Courses/CS 491ab/Winter 2008/Jonathan Berney

From CSWiki

Jump to: navigation, search

Hello, I am Jonathan Berney. This is my CS 491AB project page. I can be reached by emailing jberney(at)gmail.com. To learn more about me, please visit my User page: User:Jberney

Contents

[edit] Week 1 - January 4, 2008

We were introduced to the course, and introduced ourselves to our classmates.

As not much went on prior to this week's meeting, I'm going to use this week's space in order to:

  • Present a proposal for my CS 491AB project
  • Keep track of Platforms that I may want to present in upcoming weeks.

[edit] Proposal

This summer, I will be the instructor of a high school level introductory java course for the Academic Talent Development Program at UC Berkeley. I believe it is clear that virtual learning environments can be very helpful in computer-based courses, and so have decided to develop a general purpose one using Tomcat, servlets, and JSP that I would like to use in my course this summer.

I've worked with a few virtual learning environments in my time as a university student, such as WebCT and Professor Sun's CSNS. I was introduced to another very useful utility in learning programming basics last quarter by my CS 203 professor, Kiri Wagstaff, called JavaBat. The beauty of this web app is that it separates the code that is specific to a programming exercise from all of the code that is common to all programs. This way, you can get right down to what you're trying to learn about without having to worry about "public static void main String array args", saving, opening a command line, compiling, etc. I am interested in building this sort of functionality directly into my project.

In addition to the JavaBat-like functionality, I plan to implement many other common features of a VLE, such as course management, course announcements, a syllabus repository, a method by which to submit document based assignments, grade management/reporting, chat room, forums, and an overall calendar view. If time permits, I may look into additional features such as online quizzes where an instructor could design any particular question to be either multiple choice or short answer.

[edit] Platforms

ANT

[edit] Week 2 - January 11, 2008

[edit] This Week (2)

[edit] Apache Ant (ant.apache.org, en.wikipedia.org)

[edit] What Ant is

Ant is an open source tool that assists developers in automating source code compilation, packaging compiled code, and deployment. Unlike other similar build tools, Ant is not shell-based. Build files for Ant are written in XML, and are therefore platform-independent. The Ant executable runs on any machine with the Java SDK installed. It can be extended by writing Java classes.

[edit] How to install Ant

Binary distributions can be downloaded from the Ant website: Apache Ant - Binary Distributions.

But that's no fun. Let's compile the source! It can be downloaded here: Apache Ant - Source Distributions.

Download and unzip one of the archives. Open a command line prompt, navigate to the Ant folder that's just been unzipped and type the following:

Windows: 

build -Ddist.dir=<directory_to_contain_Ant_distribution> dist

Unix: 

sh build.sh -Ddist.dir=<directory_to_contain_Ant_distribution> dist

For instance, on Mac OS X, I typed: 

sudo sh build.sh -Ddist.dir=/usr/local/ dist

When I did this, I was greeted with the following error: 

BUILD FAILED
/Users/jberney/Downloads/apache-ant-1.7.0/build.xml:1105: The following error occurred while executing this line:
/Users/jberney/Downloads/apache-ant-1.7.0/build.xml:911: We cannot build the test jar unless JUnit is present,
      as JUnit is needed to compile the test classes.

In order to deal with this error, I went to the Apache Ant - Frequently Asked Questions page and discovered the following: 

When building Ant 1.7.0 from the source release without junit.jar the build
fails with the message "We cannot build the test jar unless JUnit is present".

With Ant 1.7.0 we've started to add ant-testutil.jar as part of the distribution
and this causes a hard dependency on JUnit - at least in version 1.7.0.
Unfortunately the installation docs don't say so.

There are two workarounds:

1. Add junit.jar to your CLASSPATH when building Ant.
2. Change Ant's buildfile and remove test-jar from the depends list of the
dist-lite target.

As I was interested in getting a working copy of Ant up and running, I opted to simply modify the Ant buildfile. In the future, I may choose to look into investigating JUnit. So I navigated to the Ant directory, opened build.xml, and changed the following code from: 

<target name="dist-lite"
    depends="jars,test-jar"
    description="--> creates a minimum distribution to run Apache Ant">

to: 

<target name="dist-lite"
    depends="jars"
    description="--> creates a minimum distribution to run Apache Ant">

(by removing test-jar from dist-lite's dependencies, as instructed).

Now, by running the command that I listed above, I was able compile and install ant. You can use 

ant -version

to make sure that it was properly installed. My copy says: 

$ ant -version
Apache Ant version 1.7.0 compiled on January 10 2008
[edit] How to use Ant

In order to use Ant, it is necessary to create an XML-based build file. A build file has one project and at least one default target. A target is made up of a set of task elements. In order to demonstrate the construction of a build file, I am going to create a project called HelloWorld. I'll do this by creating the following two files:

  • <wherever>/HelloWorld/src/HelloWorld.java
  • <wherever>/HelloWorld/build.xml

The contents of HelloWorld.java are as follows:

[edit] HelloWorld.java
public class HelloWorld {
  public static void main(String[] args) {
    System.out.println("Hello World!");
  }
}

I will start off my build.xml file by defining a project: 

<project name="HelloWorld" default="dist" basedir=".">
</project>

I have named my project HelloWorld, set the default build target to "dist" (I'll talk more about build targets shortly), and the base dir will be the directory that the build.xml file resides in. In order to define where I want my compiled work to go, I will set some properties of the project: 

<project name="HelloWorld" default="dist" basedir=".">

  <property name="src" location="src"/>
  <property name="build" location="build"/>
  <property name="dist" location="dist"/>

</project>

Next, I want to start defining some build targets. I have decided that compilation for my project will take place in two steps:

  1. Compilation (I will call this phase "compile")
  2. Jarring the class files (I will call this phase "dist")

I will define my compile target as follows: 

<project name="HelloWorld" default="dist" basedir=".">

  <property name="src" location="src"/>
  <property name="build" location="build"/>
  <property name="dist" location="dist"/>

  <target name="compile">
    <mkdir dir="${build}"/>
    <javac srcdir="${src}" destdir="${build}"/>
  </target>

</project>

This target uses the mkdir tag in order to create my build directory, where my class files will go. Then, the javac tag is used to build the source code and stick the class files in the build directory.

Next, I will write my dist target: 

<project name="HelloWorld" default="dist" basedir=".">

  <property name="src" location="src"/>
  <property name="build" location="build"/>
  <property name="dist" location="dist"/>

  <target name="compile">
    <mkdir dir="${build}"/>
    <javac srcdir="${src}" destdir="${build}"/>
  </target>

  <target name="dist" depends="compile">
    <mkdir dir="${dist}/lib"/>
    <jar jarfile="${dist}/lib/HelloWorld.jar" basedir="${build}"/>
  </target>

</project>

In addition to naming the target, I have asserted that the dist target depends on compile being run. This way, I can simply call the dist target, and if my code is not compiled, it will be done for me automatically. Anyhow, my dist target begins by using mkdir to create the dist directory (and a lib subdirectory for the jar file I will be making). Then, it uses a jar tag in order to take the classes from ${build} and archive them into a jar located at ${dist}/lib/HelloWorld.jar.

I may also wish to write a clean target: 

<project name="HelloWorld" default="dist" basedir=".">

  <property name="src" location="src"/>
  <property name="build" location="build"/>
  <property name="dist" location="dist"/>

  <target name="compile">
    <mkdir dir="${build}"/>
    <javac srcdir="${src}" destdir="${build}"/>
  </target>

  <target name="dist" depends="compile">
    <mkdir dir="${dist}/lib"/>
    <jar jarfile="${dist}/lib/HelloWorld.jar" basedir="${build}"/>
  </target>

  <target name="clean">
    <delete dir="${build}"/>
    <delete dir="${dist}"/>
  </target>

</project>

This task simply recursively deletes the build and dist directories.

Now we're ready to try it out! Navigate to the HelloWorld directory on the command line. Once in the same directory as build.xml, simply type ant. If everything is okay, you will see something like this: 

$ ant
Buildfile: build.xml

compile:

dist:

BUILD SUCCESSFUL
Total time: 0 seconds

A jar file will be created in the expected location (HelloWorld/dist/lib/HelloWorld.jar). Of course, this jar file isn't going to work since I didn't bother writing a manifest file for it.

Now it's time to clean up. Type "ant clean" at the command line to accomplish this: 

$ ant clean
Buildfile: build.xml

clean:
   [delete] Deleting directory /Users/jberney/Desktop/ant-test/build
   [delete] Deleting directory /Users/jberney/Desktop/ant-test/dist

BUILD SUCCESSFUL
Total time: 0 seconds

A list of many of the task tags that can be used with Ant is available at ant.apache.org.

[edit] Overall Progress (2)

So far, I have put together a statement of purpose, an outline of my project's structure, and a development schedule.

I have also begun to investigate working with the Ant build tool, which I will use to automate the building process.

Additionally, I've been learning that I will have some teacher's assistants this summer, so I will need to update my outline to accommodate a new permission level for this role.

[edit] Next Week (2)

By next week, I would like to:

  1. Work a Teacher Assistant role into the project
  2. Get started on 0.1.x. I would like to complete 0.1.0 if time permits.

Note: for the record, the version numbering system I'm using is:

  • First digit: Major revision - large changes (0 = development release, 1 = first major release, 2 = second major release, etc)
  • Second digit: Minor revision - feature additions (1 = first minor revision, 2 = second minor revision, etc)
  • Third digit = Bug fix - as the name implies, bug fixes

[edit] Week 3 - January 18, 2008

[edit] This Week (3)

This week, I made the following progress:

[edit] Battled influenza!

Unfortunately I got to spend more time this week thinking and dreaming about my project than working on it.

[edit] Acquired and setup Tomcat

Very simple. I found some very helpful configuration tips and tricks at coreservlets.com, a website by the authors of our CS 320 textbook.

[edit] Established a simplified deployment method with Ant

build.xml: 

<project name="SAGE" default="dist" basedir=".">

    <property name="src" location="src"/>
    <property name="build" location="ROOT/WEB-INF/classes"/>
    <property name="dist" location="/usr/local/tomcat/webapps/SAGE/"/>

    <target name="dist" depends="compile">
        <mkdir dir="${dist}"/>
        <copy todir="${dist}">
            <fileset dir="ROOT"/>
        </copy>
    </target>

    <target name="compile">
        <javac srcdir="${src}" destdir="${build}"/>
    </target>

    <target name="clean">
        <delete dir="${dist}"/>
    </target>

</project>

This buildfile compiles my java code, sticks it in my development directory's WEB-INF/classes directory, and copies my entire development directory to my tomcat installation in /usr/local/. The result is very convenient, so my research last week has been put to good use.

[edit] User Registration

First, I wrote the HTML form to take in the user registration data.

Next, I wrote a servlet (org.jberney.webapps.SAGE.registration.RegistrationServlet) to process this form. Initially the servlet simply checked for valid data--are the names alphabetical characters only, does the email address look valid, is the login name all alphanumerics, is the password long enough, etc--and forwarded the user to a "thank you for registering" page if everything looked good. Then I wrote a bit more to allow the user to try again if there were errors, with error feedback.

Finally, I had to get my hands dirty with the database access. I wanted to be very careful with this, in that I wanted to avoid hardcoding too much of the connection initialization information, such as the username, password, and driver name into the java source file. I discovered a technique called connection pooling, which would allow me to define all of these values in a static xml file. The value of this for me is that it will allow me to distribute compiled code where the database specifics can be configured by an end user.

Accomplishing this was very tricky, as it seems that most tutorials are written for Tomcat 4. The result was not particularly complicated in the end though. I had to add to my /WEB-INF/web.xml file: 

<resource-ref>
  <res-ref-name>jdbc/sage</res-ref-name>
  <res-type>javax.sql.DataSource</res-type>
  <res-auth>Container</res-auth>
</resource-ref>

The tricky part, in that it differed for me due to my having a newer version of Tomcat than that for which most tutorials are written, was creating a file at /META-INF/context.xml with the following content: 

<?xml version="1.0" encoding="ISO-8859-1"?>
<Context>
  <Resource name="jdbc/sage"
            auth="Container"
            type="javax.sql.DataSource"
            maxActive="100"
            maxIdle="30"
            maxWait="10000"
            username="(username)"
            password="(password)"
            driverClassName="com.mysql.jdbc.Driver"
            url="jdbc:mysql://localhost:3306/(dbname)" />
</Context>

The Users table I currently am using to keep track of registered users was created with: 

create table Users(
    userID mediumint,
    role tinyint,
    firstName varchar(32),
    lastName varchar(32),
    emailAddress varchar(32),
    loginName varchar(32),
    loginPassword varchar(32),
    hue tinyint,
    saturation tinyint
);

Once my web app had database access, I was able to use it ensure that each user has a unique userID. I still need to ensure unique login names, work on masking passwords, and I'd like to have registration and login/authentication happen over SSL (https).

[edit] Overall Progress (3)

A live version of my project is available at sage.endofinternet.org

I have focused so far on functionality over looks. With that in mind, user registration is nearly done. I still need to check for duplicate login names so that there are not any duplicates.

[edit] Next Week (3)

Next week, I would like to complete the items that I have listed for 0.1.0 (all of the very basic actions for the users of each permission level).

[edit] Week 4 - January 25, 2008

[edit] This Week (4)

[edit] Registration Part 2

User registration is now essentially complete. The three outstanding issues that remained last week were:

[edit] Duplicate login names were allowed

While users are uniquely identified in my database by their user id number, it's important that no two users share the same login name. Fixing this was a fairly straightforward programmatic problem. 

private boolean isUnique(String input) {
    String inputLower = input.toLowerCase();
    DatabaseUtil du = new DatabaseUtil(dataSource);
    
    ResultSet rs = du.executeQuery("select sageID from Users");
    
    try {
        while(rs.next()) {
            String current = rs.getString(1).toLowerCase();
            if(current.equals(inputLower)) return false;
        }
    } catch(Exception e) {
        return false;
    }
    return true;
}
[edit] Passwords were stored as-is in my database

Even if it's not good practice, users should feel safe using passwords that they commonly use for other purposes. It would be a security concern if passwords for this site are stored as-is in the database that a database administrator could simply look up. In order to solve this problem, I obtained a javascript implementation of the SHA-1 algorithm from a website called Paj's Home. I use a SHA-1 function as follows: 

function hash(...) {
  var fullPassword = "SAGE " +
                     fixName(form.sageID.value) + " " +
                     form.password1.value.length + " " +
                     form.password1.value;
  form.passhash.value = hex_sha1(fullPassword);
  ...
  return true;
}

This way, the user's password is never sent from the user's computer to the web application. Instead, a 40-character hash of the password (concatenated with some other things, as shown in the javascript excerpt above) is generated client-side by the user's browser, and it is this hash that is sent to the web app and stored in a database. Even if an administrator were to see this hash, or anyone else for that matter, it would be nearly impossible to decrypt this hash to discover the password originally chosen by the user.

[edit] Passwords were sent as-is over the internet without encryption

This problem is currently only half-solved. The first part of the problem is that the password was being sent as-is over a connection that could easily be packet-sniffed. This would allow anyone listening to discover the user's password. Now that the password is hashed and never sent, the most a malicious user could learn is the password hash. This is still enough for someone other than the user to log in as the user, but at least the actual password is never learned. In order to fully solve this problem, I will have to investigate SSL connections, as I discussed briefly last week. I have decided not to worry about this issue for now, and come back to it at a later time.

[edit] Index redirection

The web app's index page is now a portal of sorts that directs a user to the appropriate location based on their permissions/roles: 

<jsp:useBean id="sessionBean"
             class="org.jberney.webapps.SAGE.session.SessionBean"
             scope="session" />

<%
String destination = "/WEB-INF/jsp/" +
sessionBean.getRole() +
"/index.jsp";
%>

<jsp:forward page="<%= destination %>" />

When a user first arrives, they are directed to the SesssionBean's default role location, the anonymous index. This index presents the user with the option to either register as a student, or log in with existing credentials. When a user chooses to log in, the SessionBean's role (and other relevant information) is updated to reflect the role of the user. Then, when they are redirected back to the index portal, they will be pointed to the appropriate location based on their permissions set.

[edit] Login

In order to log into the web application, a user is required to provide their login name (which I am now calling the Sage ID) and their login password. The password is hashed with a javascript, as it is for the registration form. The form info is then passed along to a servlet, which first checks that the sageID exists, and then that the hashed password supplied by the form matches the hash in the database. Assuming that both of these are true, some of the user's data (first name, last name, userid, etc) is stored in a SessionBean for later use. Finally, the user is redirected back to the index, where they are again redirected to a page based on their role as a student, teacher's assistant, instructor, or administrator.

[edit] A little work on the look of the site

A picture says a thousand words:

Image:Sage070125.gif

[edit] Overall Progress (4)

User registration and login are essentially complete. So, I am done with the anonymous tasks. This leaves basic administrator, instructor, TA, and student tasks before I am done with v0.1.0, as noted on my outline.

[edit] Next Week (4)

My work over the next week will begin with implementing administrator tasks. Then I will work on instructor tasks, followed by TA tasks, and student tasks.

Also, more generally, I need to decide on a basic page layout for the site--issues such as where navigation links will go, whether to use a tree structure for them or require users to click through pages, etc.

[edit] Week 5 - February 1, 2008

[edit] This Week (5)

[edit] Moodle!

[edit] Installation

Much to my delight, I am saved from having to write much of the tedious functionality of a Virtual Learning Environment. Enter Moodle, the Modular Object-Oriented Dynamic Learning Environment. Setup was a snap, as I already had all of the required stuff up and running on my computer. My still rather bare-bones installation of Moodle is available here.

[edit] What I will be doing with Moodle

The M in Moodle stands for modularity, and from just looking at the installation's directory structure, it is very obvious as to why this is. It appears that I can write an Assignment plugin that would require the student to enter some java code, have it compile in the browser, show the stdout, as well as a number of tests that would be pre-defined by the instructor. Unfortunately, the developer documentation for developing new Assignment-based plugins is non-existent. Nevertheless, I believe it will be easy enough to just look at what's been done to get an idea of how to do what I would like to do.

[edit] PHP and Java

[edit] PHP

It is with some reluctance that I transition into doing this project in entirely PHP. I am not particularly familiar with the language, I know and love Java, and most comparisons I read between the two result in Java being the language of choice for web applications. Nevertheless, this is a good opportunity to learn PHP, and Moodle appears to be an excellent product that has been developed and tested by many people in real-world environments.

[edit] Getting Java into PHP

Originally, I had intended to use Java SE6's JavaCompiler class with my web application. The benefit of this is obvious: cross-platform java compilation from within a program. No worrying about where javac lives, what operating system the server is running, etc. Now that I am going to be working in a PHP environment, I will need to see how PHP can get at Java in order to access this JavaCompiler class.

I am running PHP5 on my webserver. From http://www.php.net/java I learned that with a simple recompile of PHP, I could essentially call Java functions. But when I tried to configure the PHP compile, I was told that --with-java was no longer a legal argument. Searching around on the web, I learned that the official PHP-Java bridge has been removed starting with PHP5. Unwilling to revert to PHP4 for this functionality, I began searching for other solutions.

With some more searching, I found an open source Java-PHP bridge at sourceforge. When I downloaded it, I discovered that a requirement of the package was that it be run in a servlet environment. As mixing the two technologies, Servlets and PHP, sounds like a bit of a headache, this is also not a good solution for me.

Instead of the above solutions, I have decided that it may be best to directly call javac from the PHP with exec() function. This way, I do not need to require Java SE6, and I can stick to just php. The downside is that I have to make platform-dependent system calls that need to know where javac and java live on a server.

[edit] Compiling HelloWorld.java from a PHP Script

For this example, I will be reusing my HelloWorld.java file that I wrote in Week 2.

First, I need to create a class that will be responsible for compiling the java code. I will call it JavaCompiler (javaCompiler.php): 

<?php

/* This is a class that compiles java code. */
class JavaCompiler {

}

?>

I will need some instance variables that store the locations of the source code and built code, as well as a constructor to initialize them: 

<?php

/* This is a class that compiles java code. */
class JavaCompiler {

  var $sourcePath; // .java location
  var $buildPath;  // .class location

  /* Constructor */
  function JavaCompiler($sourcePath, $buildPath) {
    $this->sourcePath = $sourcePath;
    $this->buildPath = $buildPath;
  }

}

?>

Now it gets a little tricky. I need a method that will compile code in a given location, send it off to another location, and write error messages to a specified text file. The method should return a value that indicates if compilation succeeded. The command I would like to run then will look something like this: 

 javac -sourcepath <path-to-java-files>
       -d <path-to-class-files>
       -Xstdout <path-to-output-file>
       <whatever.java>

The compile method then takes the following form: 

<?php

/* This is a class that compiles java code. */
class JavaCompiler {

  var $sourcePath; // .java location
  var $buildPath;  // .class location

  /* Constructor */
  function JavaCompiler($sourcePath, $buildPath) { ... }

  /* Compiles */
  function compile($className) {
    $cmd = "javac " .
           "-sourcepath " . $this->sourcePath . " " .
           "-d " . $this->buildPath . " " .
           "-Xstdout " . $this->buildPath . $className . ".txt " .
           $this->sourcePath . $className . ".java";
    exec($cmd, $output, $return);
    return $return;
  }

}

?>

I also need a method to run the code and return the console output. It will be necessary to point to where the .class files are, so I need a classpath flag: 

java -cp <path-to-class-file> <class-filename>

The run method: 

<?php

/* This is a class that compiles java code. */
class JavaCompiler {

  var $sourcePath; // .java location
  var $buildPath;  // .class location

  /* Constructor */
  function JavaCompiler($sourcePath, $buildPath) { ... }

  /* Compiles */
  function compile($className) { ... }

  /* Runs */
  function run($className) {
    $cmd = "java -cp ".$this->buildPath." ".$className;
    exec($cmd, $output, $return);
    return $output;
  }

}

?>

I need a function that will clean out the build and src directories. For the time being, I am not actually wiping the src directory, as I am not magically receiving random source code files yet. So to just wipe the build directory, I need something like: 

rm <path-to-build-files>

And the clean function is: 

<?php

/* This is a class that compiles java code. */
class JavaCompiler {

  var $sourcePath; // .java location
  var $buildPath;  // .class location

  /* Constructor */
  function JavaCompiler($sourcePath, $buildPath) { ... }

  /* Compiles */
  function compile($className) { ... }

  /* Runs */
  function run($className) { ... }

  /* Cleans */
  function clean($className) {
    unlink($this->buildPath.$className.".class");
    unlink($this->buildPath.$className.".txt");
  }

?>

Lastly, I need a function that will open the output text file in case any errors have occurred during compilation: 

<?php

/* This is a class that compiles java code. */
class JavaCompiler {

  var $sourcePath; // .java location
  var $buildPath;  // .class location

  /* Constructor */
  function JavaCompiler($sourcePath, $buildPath) { ... }

  /* Compiles */
  function compile($className) { ... }

  /* Runs */
  function run($className) { ... }

  /* Cleans */
  function clean($className) { ... }

  /* Returns compile errors */
  function getErrors($className) {
    $compileErrors = $this->buildPath . $className . ".txt";
    $handle = fopen($compileErrors, "r");
    $errors = fread($handle, filesize($compileErrors));
    fclose($handle);
    return $errors;
  }
 
?>

And of course, a fairly self-explanatory test case (index.php): 

<?php

// Requries the JavaCompiler class
require_once "javaCompiler.php";

// This is the class name to compile. It will not be hard-coded.
$className = "HelloWorld";

// Required paths.
$sourcePath = "/usr/local/apache2/java/src/";
$buildPath = "/usr/local/apache2/java/bin/";

// Instantiate a JavaCompiler object.
$compiler = new JavaCompiler($sourcePath, $buildPath);

// Compile the java code and store the result in a variable.
$result = $compiler->compile($className);

// If compilation was successful, run the bytecode.
if($result == 0) $compiler->run($className);

// Otherwise, show the compile errors.
else echo $compiler->getErrors($className);

// Clean things up.
$compiler->clean($className);

?>

[edit] Overall Progress (5)

My work prior to this week, while very interesting and educational, is mostly no longer useful. I will no longer need ANT for building and deploying my code, as PHP is an interpreted language and does not need to be compiled. I will need to rework my plan. This week was all about installing Moodle, getting a feel for it, and seeing if it would even be possible to get PHP to compile java code and have access to the output.

[edit] Next Week (5)

Next week I would like to:

  • Read more about Moodle plugin development and hopefully build something like a hello world plugin for it.
  • Take the php code I have now and figure out how I'm going to get from the form input to:
    • Insert simple methods into a .java file that will have the tedious elements automatically generated (public class <classname>, constructor, public static void main(), etc, these kinds of things)
    • How will instructors define the tests (which will be used to determine if a student passes/fails an assignment, or what grade they receive, etc), what packages are imported, or is this left up to the student?
    • How to restrict whatever needs to be restricted for security purposes, or will this be taken care of by the web server and operating system?
    • Show the output (sort of already done)
    • Show the output of the tests
  • Plan out all of what I would like to accomplish for this project, and come up with a new schedule for the tasks ahead.

[edit] Week 6 - February 8, 2008

[edit] Last Week (6)

Just to recap, last week, I managed to get php to call javac in an essentially platform independent way: Experiment #1: HelloWorld.

[edit] This Week (6)

[edit] Experiment #2: RandomClassName

Friday after class, I got home and got to work. My first goal was to make a template class called RandomClassName. This class would be sort of a template that would allow an instructor to specify certain parameters, such as what Java packages to import, the solution to a given exercise, the test methods, and test arguments. The student's code would be filled in when the student submits an assignment. An example of the output is here: RandomClassName example.

[edit] Experiment #3: DisplayTable

With RandomClassName working, my next goal was to come up with a display table of the results to be shown after a student submits their code. I made a sample page that shows the results of a Recursion exercise from

  • a student who has the correct solution,
  • a student with an incorrect sotluion, and
  • a very simple solution by an instructor

The experiment is available here: DisplayTable.

[edit] But wait, there's more: Moodle 1.8.4

Last week, I installed Moodle 1.8.4 on my computer. I got to work implementing a java assignment type, and finished much quicker than I anticipated. Rather than paste a lot of code, I'd like to demonstrate the process. I will leave some screenshots here, and actually step through the process as I present in class.

[edit] Had to do bad stuff to make it work

In order to get the module to function 100%, I had to do some editing of some core code files. This sort of goes against the whole idea of modularity. So I posted to the developer discussion, and within a few hours got a reply from the lead developer of Moodle, who informed me that the beta release Moodle 1.9 would solve my problems.

The result: it was a headache, and only solved some of my problems. But the problems it did solve, it solved well, and allowed me to do some nifty things that weren't readily available in my previous implementation:

[edit] The end?

I'm certainly not done, but I don't think this project is something that will last me the next four months, so early last week I decided that I needed something new to work on.

Yesterday evening, after I got home from a midterm, one of my web design friends had an idea for a project for me that I would like to pursue over the next quarter and a half.

[edit] A decade or two ago

Back in the early days of website design, most sites were simply white background, black text, blue links, purple when visited, etc. Over the last decade, web designers have become much more liberal in the use of color on websites. You don't have to look far for examples of this, CSS Zen Garden for instance: css Zen Garden Design List. An interesting Adobe.com web app called Kuler has order to aid web designers in coming up with color schemes. There are several presets, but the most interesting feature of the site is the Create option. I will demonstrate the function in class, but briefly, it allows you to first come up with a color scheme based on a rule, and then by moving levers on a color wheel, shift the hue and saturation of the entire color scheme.

The problem with this website is that once a designer finds a color scheme that MIGHT work, they must go to their css documents, find/replace hex colors in several locations, switch to their browser, refresh, discover it doesn't work, rinse and repeat. Add to that the fact that with each iteration they may have to hue shift all of the images on their site in order to get a feel for a particular color scheme.

The project I propose would be to create an application that implements the rule-based color wheel and allows a web designer to see their website change live in reaction to their interactions with the color wheel.

I think it would be very interesting to accomplish this with Google's Google Web Toolkit. This platform allows me to write an AJAX web application using Java. I am not sure, however, that it would work. If I look into this and it is not the case, my second choice would be to create a Java desktop application with the Standard Widget Toolkit, which I have used extensively over the past few months in order to create my IoChatClient.

[edit] Overall Progress (6)

My work with the java exercises is mostly done. I suspect I'll continue to refine it over the next few months before my class begins, but I do not believe that it is enough to constitute my CS491 project at this point. If there is a good argument against my moving to the new project that I have proposed, I can probably come up with ways of making my Moodle project larger and more interesting, such as adding support for uploaded source code files, groups of source code files, more languages, etc.

If I do end up switching to my newly proposed project, I certainly have a lot of work ahead.

[edit] Next Week (6)

By next week, I would like to do a full investigation of the capabilities of GWT and SWT to see how feasible each would be for my new project idea. Mainly, the two goals I am interested in are:

  1. Can the toolkit, based on user input, do a reasonably live update of the colors of html elements in a CSS-based webpage
  2. Can the toolkit, based on user input, do a reasonably live update of the colors of images in a webpage

By "reasonably live," I would prefer that the colors update as the user moves the bars around the color wheel, but would settle with colors changing once the mouse button is lifted.

[edit] Week 7 - February 15, 2008

[edit] This Week (7)

My challenge this week was to think of ways in which I can improve my Java-Moodle module. Here were some ideas that were given to me (or that I had) and why I may or may not want to pursue them.

[edit] Interactive Java Interpreter

An interactive java interpreter would, I assume, allow for the compilation and tests to be truly live within the browser. Involving the server in this would be wasteful of network resources, and would presumably introduce unwanted latency, so this would probably be best implemented as a client-side browser-based javascript. There are a few reasons why this idea doesn't appeal to me, based on my perception of its implementation:

  • May be disruptive to some
  • Requires the client to have the JDK installed
  • No access to the instructor solution, and so tests cannot be run
  • Live compilation discourages hitting the save button, so progress may be lost if the user accidentally closes the browser

[edit] Code Analysis

It's possible that my php code could analyze submitted source code and generate suggestions based on errors. However,

  • I have no idea how this would be done in a practical way
  • I hesitate to give students any additional output than what javac would provide, because it may make them spoiled.

The goal of the module as I see it is to streamline assignment document saving, compilation, testing, submission, and grading -- to assist with things that are not directly a part of the programming exercise itself.

[edit] Graduated Collection of Exercises

This would make some sense to do, because:

  • It would be good to include with the module
  • I need it for my summer course anyway

[edit] Improve Modularity

Right now, my module has the strict requirements that the instructor use my template for automatically generated Java classes. Additionally, only Java programs are allowed. There's no reason I couldn't make this more modular and allow instructors to specify their own template file (and maybe include some default ones), and allow instructors to choose the programming language the assignment should be written in.

[edit] Fix the core code changes

As stated last week, I had to change some "core code" in order to get everything to work 100% correctly. In order to fix this, I will have to either

  • Figure out how to do these things correctly, if it's even possible to do so (doesn't seem to be the case, but I should investigate further)
  • Fix the core files to be more modular
  • Branch the assignment type that I've written to be a complete module separate of the assignment module. This would make the problems I've had simple to fix, but would involve a lot of arguably unnecessary code rewriting.

[edit] Branch the assignment type to be a separate module

Perhaps I could make an entirely new module. This would give me more flexibility and allow me to do some neat stuff that is very cumbersome to do as an "assignment type" such as make different categories of programming assignments:

  • Exercise: Basically what I've got now
  • Applet: What I have now, with graphical output
  • Full file: Write an entire class. Right now my work is limited to implementing methods in a class.
  • Project: Add/Edit/Delete class files.
  • Upload: Would be like assignment upload, but with automated grading.

[edit] Week 8 - February 22, 2008

[edit] This Week (8)

This week, I have decided to expand my project from an browser-based java exercise system into a complete module for Moodle that will allow fairly arbitrary programming assignments.

[edit] Language independence

First, I would like to make what I have done so far work for any programming language, including C, C++, Prolog, etc. I hope to make it very extensible so that support for additional languages can be easily added. I might choose to add modules for some additional languages myself, depending on interest and time constraints, but my focus will still be primarily on developing Java functionality.

[edit] Required components for each language type

For each language type added to my module, there will need to be some required components. The first component will be a php-based way of allowing the php scripts to communicate with the specific language's compiler/interpreter/etc. The second component will be a folder containing different modules which correspond to different types of assignments that can be created for a particular language. More detail below:

[edit] Bridge from PHP

In Week 5, I first began work on bridging PHP and Java (see: PHP and Java). For every language that gets included, this sort of thing will need to be in place for any kind of assignment type that could be created. So this will go in the base directory of the language, to be used by each assignment type.

[edit] Types of assignments

In Week 6, I wrote code for what will be a Java language Exercise assignment type in the new scope of my project (see: Java language Exercise assignment type).

Everything that is specific to a particular assignment type will go in the "type" directory inside of a language folder. For example, the files specific to my Exercise assignment type would (tentatively) be located at $moodledir/mod/programming_assignment/programming_language/java/type/exercise/. The kinds of files that would go here would be:

  • Code evaluator: calls the base java library php to compile code, run tests, and return results
  • Template file: Text file that will have the instructor and student solutions inserted with a string formatter
  • Role-based web view / stylesheet: php that describes how everything will be displayed in a browser
  • Bank of included default assignments: Some defaults to use with the assignment type. Homework assignment packages can be shared among instructors and dragged/dropped into this location.
  • Database definitions: Instructions for Moodle as to how to store relevant info in a database
  • Language specific strings: I plan on only doing US English strings, but would allow other languages to be supported

[edit] Preliminary directory layout

Image:Pa_sample.jpg

[edit] Overall Progress (8)

So far, I have:

  • Developed a fairly concrete idea of what my project will be: a Moodle module that will allow the automated submission and evaluation of programming assignments.
  • Come up with a tentative structure for the project (this week, see Pa_sample.jpg).
  • Gotten a good feel for Moodle module development by creating a Java assignment type (see But wait, there's more).
  • Bridged PHP and Java (to the extent that I require, see my experiments).

An interesting observation is that while the above list is logically organized from the most abstract to the most concrete, it was in the reverse order that I accomplished each item, but I guess that's how experimentation works!

[edit] Next Week (8)

My goals for next week:

  • Finalize project goals
  • Create a development schedule
  • Work on my final report
  • Start coding! Specifically, my first goal will probably be to get the functionality I've developed so far into the new model that I've come up with.

[edit] Week 9 - February 29, 2008

[edit] This Week (9)

[edit] CS320

Between now and the end of the quarter, I have to complete a project for CS320. A lot of my time this week went towards working on this project, and it's going well (here's the current API). As this project is due in just a few weeks, I decided to give it a bit more priority this week than my Moodle module.

[edit] Final Report

I have completed a draft of my final report.

[edit] "newmodule"

With some searching around the Moodle developer forums, I found a bare-bones Moodle module template package. This will give me a good starting point. I installed it as it was and learned about how module installation is done--aside from dragging and dropping a third party module into the correct location, it's not immediately obvious as to how you get the database tables written, as well as other things that are required for a successful installation.

Here's the sample install.xml file that comes with the newmodule package: 

<?xml version="1.0" encoding="UTF-8" ?>
<XMLDB PATH="mod/newmodule/db" VERSION="20070401" COMMENT="XMLDB file for Moodle mod/newmodule"
    xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
    xsi:noNamespaceSchemaLocation="../../../lib/xmldb/xmldb.xsd">
  <TABLES>
    <TABLE NAME="newmodule" COMMENT="Default comment for newmodule, please edit me">
      <FIELDS>
        <FIELD NAME="id" TYPE="int" LENGTH="10" NOTNULL="true" UNSIGNED="true" SEQUENCE="true" ENUM="false" NEXT="course"/>
        <FIELD NAME="course" TYPE="int" LENGTH="10" NOTNULL="true" UNSIGNED="true" DEFAULT="0" SEQUENCE="false" ENUM="false" COMMENT="Course newmodule activity belongs to" PREVIOUS="id" NEXT="name"/>
        <FIELD NAME="name" TYPE="char" LENGTH="255" NOTNULL="true" SEQUENCE="false" ENUM="false" COMMENT="name field for moodle instances" PREVIOUS="course" NEXT="intro"/>
        <FIELD NAME="intro" TYPE="text" LENGTH="medium" NOTNULL="false" SEQUENCE="false" ENUM="false" COMMENT="General introduction of the newmodule activity" PREVIOUS="name" NEXT="introformat"/>
        <FIELD NAME="introformat" TYPE="int" LENGTH="4" NOTNULL="true" UNSIGNED="true" DEFAULT="0" SEQUENCE="false" ENUM="false" COMMENT="Format of the intro field (MOODLE, HTML, MARKDOWN...)" PREVIOUS="intro" NEXT="timecreated"/>
        <FIELD NAME="timecreated" TYPE="int" LENGTH="10" NOTNULL="true" UNSIGNED="true" DEFAULT="0" SEQUENCE="false" ENUM="false" PREVIOUS="introformat" NEXT="timemodified"/>
        <FIELD NAME="timemodified" TYPE="int" LENGTH="10" NOTNULL="true" UNSIGNED="true" DEFAULT="0" SEQUENCE="false" ENUM="false" PREVIOUS="timecreated"/>
      </FIELDS>
      <KEYS>
        <KEY NAME="primary" TYPE="primary" FIELDS="id" />
      </KEYS>
      <INDEXES>
        <INDEX NAME="course" UNIQUE="false" FIELDS="course"/>
      </INDEXES>
    </TABLE>
  </TABLES>
  <STATEMENTS>
    <STATEMENT NAME="insert log_display" TYPE="insert" TABLE="log_display" COMMENT="Initial insert of records on table log_display. Each record describes how data will be showed by log reports.">
      <SENTENCES>
        <SENTENCE TEXT="(module, action, mtable, field) VALUES ('newmodule', 'add', 'newmodule', 'name')" />
        <SENTENCE TEXT="(module, action, mtable, field) VALUES ('newmodule', 'update', 'newmodule', 'name')" />
        <SENTENCE TEXT="(module, action, mtable, field) VALUES ('newmodule', 'view', 'newmodule', 'name')" />
      </SENTENCES>
    </STATEMENT>
  </STATEMENTS>
</XMLDB>

This xml file describes the actions Moodle should take when the module is first installed, with regards to creating new database tables. By itself, it's just a bunch of text. In order to actually get Moodle to look at the text and create the tables, an administrator needs to:

  • Put the module in the correct location ($moodledir/mod/).
  • Go to Notifications in the Site Administration table. This will create the necessary DB tables and show any associated output, errors, etc. Without doing this step, the database tables are not created.
  • Go to Modules -> Activities -> Manage Activities to verify that the module has been installed.

[edit] Week 10 - March 7, 2008

[edit] Moodle Module Anatomy

This week I investigated the files common to every Moodle module in order to figure out for certain what exactly everything does.

Moodle modules use the MVC (Model View Controller) approach. Only the model and the view need to be taken care of, as controller functions are implemented by core Moodle code.

[edit] View

[edit] icon.gif

The icon.gif file is used in several locations to give a visual indicator of a module. The icon I am using is the Xcode icon of a C++ document:

Image:cppicon.jpg

[edit] index.php

The index.php file lists all instances of the module in a course. For my module, it will be a listing of all of the programming assignments that have been created.

[edit] view.php

The view.php file is a page to view a particular instance of the module. In my case, this is where students will be able to view, edit, and submit their work, as well as view immediate feedback with regards to compilation errors, logic errors, or success.

[edit] mod_form.php

The mod_form.php file, despite its general sounding name, is just a form for configuring instances of the module. This file will be what allows instructors to initially configure a programming assignment, and later edit it.

[edit] other php files

There will definitely be more View-based php files to write. For instance, like the assignment module, I will write a submissions.php page so that an instructor can view and grade all of the student submissions for a particular assignment at a glance.

[edit] Model

[edit] lib.php

The lib.php file is a library defining all of the logic behind a module. It has several required functions, which have to do with module installation, instance management, and cleanup. Optionally, the logic that is not common to and required of all modules can be placed in a file called locallib.php. I will probably do this, to keep things clean. Allegedly it can also improve performance.

[edit] db, version.php

The version.php file defines the version of the project. The version number is conventionally a datestamp of the time at which the module was last updated. This is used by database files to determine when database structure needs to be updated. How the database structure is updated is defined in db/upgrade.php.

[edit] install.xml

The install.xml file is an XML file that defines what tables Moodle needs to create for a module.

[edit] Demonstration!

(In-class demonstration of my java assignment type and new program module, showing where each file is getting called in my installation of Moodle)

[edit] Final Report

[edit] Brief project description

My project is a module for Moodle that allows instructors to create programming assignments that can be automatically evaluated for correctness with visual results as a student works on them.

[edit] Anticipated users

This module could be useful for anyone who is teaching or taking a programming class, particularly for introductory courses.

[edit] Main conceptual (i.e., user-level) objects

The objects that concern the users (instructors and students) are the programming assignments that the instructors will create, the students will work on, and the instructors will grade. Many pages in the system will show lists of these assignments and allow certain operations, which are discussed in the next section.

[edit] Primary conceptual (i.e., user-level) operations

  • Instructor: Create/edit/view/grade assignments
  • Student: View/edit assignment submissions

[edit] Why I am interested in this project

I was interested by the approach taken by the JavaBat website, which provides a number of java exercises which are solved and evaluated in a web browser window. It was my feeling that this sort of immediate feedback could be very helpful in teaching beginning programming students without swamping them with the overhead of a full-blown IDE.

More practically, I will be teaching an introductory course in Java for the ATDP program at UC Berkeley this summer, and was interested in developing a tool similar to JavaBat that would integrate with some common Virtual Learning Environment (VLE) features. This was the basis of my project, which has become much more general, and will use Moodle for the extra VLE features.

[edit] Status

I have:

  • Explored Moodle's VLE capabilities for use with my project
  • Bridged PHP and Java (to the extent that I require, see my experiments) Image:Displaytable.jpg
  • Gotten a good feel for Moodle module development by creating a Java assignment type (see But wait, there's more). Image:assignmenttype.jpgImage:logicerrors.jpgImage:grading.jpg
  • Come up with a structure for the project (this week, see Pa_sample.jpg).Image:Pa_sample.jpg
Retrieved from "http://cs.calstatela.edu/wiki/index.php/Courses/CS_491ab/Winter_2008/Jonathan_Berney"