Current MS Students/Nidhi Goel/CS 590
From CSWiki
Contents |
[edit] Jan 6 (week 1)
[edit] Jan 12 (week 2)
I got introduced to course objectives and discussed about course requirements with Prof. Abbott. I met with Prof. Sun to find if he had projects for master thesis. He suggested me a couple of projects.
I found a project related to web based assessment system very interesting. This project aims to analyze data for developing different methods which can assess and assist high school students in a better way as compared to a traditional system. In a traditional education system, a lot of time goes waste in proper assessment of students. Using the e-assessment system, time can be utilized properly by providing instruction and assessing students at the same time. Also, the project aims at improving the accuracy of assessment of the students.
Further details about the project will be discussed in next meeting with professor.
[edit] Jan 19 (week 3)
[edit] Jan 26 (week 4)
Business Intelligence and reporting tools
This week I spent my time on BIRT. I found that this is really very powerful tool. BIRT provides core reporting features such as report layout, data access and scripting. I worked on chart section of reports. Creating a chart using the data is very easy with the help of BIRT.
Installation Guide for BIRT:- BIRT is currently tested with the following configurations
Eclipse (an open development platform)
GEF (the Graphic Editor Framework, is an Eclipse plugin used by the BIRT UI)
EMF (The BIRT charting feature uses EMF, the Eclipse Modeling Framework)
JRE (Java Runtime Environment)
BIRT Libraries
Library features have been introduced in BIRT. In this feature required functions and report items are summarized within a library. These libraries are included to design a report. It reduces the design time and increases reusability of templates. For example, a BIRT report developer may create a complex scripted data source and then summarize it within a BIRT library. Now other members of the reporting team can use the same code for their own designs.
This feature is very useful in many cases. For example, many reports use the company's logo and if the company changes its logo, there is no need to modify it in every report. Just a change in the library will make a change in all the reports.
This diagram shows the use of libraries in a report design. A report developer can use various libraries by drag and drop in a new or existing report design.
Here are some examples which is created with the help of BIRT (I selected these examples from BIRT's website).
BIRT Integration in application Currently I am doing research on BIRT integration.
[edit] Feb 2 (week 5)
Prof. Sun suggested me to read a paper 'Addressing the testing challenge with a web-based e-assessment system that tutors as it assesses'. This week I read this paper and tried to understand this. The link of this paper is http://www2006.org/programme/item.php?id=1099
What this paper discusses
In a traditional system teachers are not able to teach and assess students at the same time. This paper hypothesizes that with the developed e-assessment (E-ASSISTment) system both things can be done simultaneously. The e-assessment system can provide valuable assessment information to keep track of student's learning over the course of an year. It also emphasizes that accurate assessment can not only be judged by whether students identify items as right or wrong but also by how much effort is required by them for solving the problem. It also assesses the factors which affect learning skills of individuals.
Why this work is important
If a teacher spends time to assess a student, it means this is a loss of valuable time which could be used for instruction. Therefore, we need a system which can test every student without wasting this time. A traditional system does not distinguish between students who got same answer no matter how they got it. Accurate assessment is needed to judge a student more accurately.
Research questions of paper are
- Does the tutoring provide valuable assessment information? - Does this continuous assessment system do a better job than more traditional forms of assessment? - Can we track student learning over the course of the year? - Can we see what factors affect student learning? - Can we track the learning of individual skills? - Can we track the learning of individual skills better if we use paper practice test results as a covariate?
[edit] Feb 9 (week 6)
This week I spent my time in research of data collection. I read the paper's algorithmic part. What type of data items authors used. How did they calculate them? How they give answers of all research questions? I got the online measures which authors are using to indicate the amount of assistance a student needs. On the basis of these meaures, authors devided problem in different sub models. After comparing these models, they got the results.
DATA
The authors collected 417 students' data who were using their e-assessment system from school year 2004 to 2005. Collected data included their MCAS scores, paper and pencil test scores and 15 online measures. A question paper contains items which are independent problems or questions. An item contains scaffolding questions (questions related to the same problem) as well.
Paper and pencil tests include two categories which are following:
● SEP-TEST- This test was administrated in Sep 2004.
● MARCH-TEST- This test was administrated in March 2005.
Online measures are described in presentation which I gave in class.
Algorithm for student assessment:
Authors describe five models which use the collected data. In the paper, Table 2 (Regression Models) shows details about them. These models have been built for assessing the students. In these models authors selected different groups of independent variables for regression. The model which assesses in good a way is selected for ASSISTment system.
Following steps are taken for implementing a model:-
Step1: Select different variables among the 15 online measures to define the model.
Step2: Calculate the coefficients for corresponding variables of a model using linear regression analysis to best match the obtained MCAS scores by the students.
Step3: Calculate R2 using the calculated coefficients and sample data of students. R2 is a measure of correlation between curve fit values obtained from regression analysis equation and the sample data.
Step4: Using R2 calculate BIC (Bayesian Information Criterion) = n*log(1-R2) + p(log(n)) where n=417 (number of samples), p=number of variables. BIC is a measure of how much information in terms of samples and number of variables is needed to get a reasonably good model. Higher BIC means more information is required, which implies the model is not very good.
The authors select the model based on the following criteria
● Uses less number of independent variables
● Has large value of R2
The value of BIC gives a good assessment for such a model. Authors found that model IV was most significant.
Conclusion:
Though model V has lowest BIC values, authors think model IV is better because it uses only six variables as compared to ten variables used in model V. Table 3 in this paper contains all included variables' coefficients calculated by regression analysis for model IV. These coefficients are useful in assessment of a student.
The following equation can be used for evaluating the score of a particular student (using coefficients of model IV and on line measures):-
Total Score = PERCENT_CORRECT*0.27 + SEP_TEST*0.45 + MARCH_TEST*0.47 – AVG_ATTEMPT*8.508 - AVG_HINT_REQUEST *2.444 + AVG_ITEM_TIME*0.021
Here is the Assistment presentation I gave in class.
[edit] Feb 16 (week 7)
I'll be volunteering for progfest on Saturday.
I spent this week in understanding mathematical concept of paper which I presented in class. In calculations, correlation and regression plays important part. I worked on correlation and linear regression. I am searching the papers which are related to this concept. After getting to know about everything, I'll be able to write something about my thesis prospectus.
Correlation
According this paper authors are trying to correlate MCAS score data points with assistment data points so that they can prove that their system can predict MCAS scores. After correlation they got the following table :
This table is not giving satisfactory result so we need to do Regression.
Simple Linear Regression
In this linear regression, we use two variables. As for this paper one variable is MCAS score and second one is any online measure. Here is the way how to do Linear regression.
[edit] Feb 23 (week 8)
This week I worked on mathmatical concept like determining coefficient and multiple regression analysis. Now I understand the concept behind this. Why we do that?
