A Note for SCContest2008

I am chairing the IEEE Service Computing Contest 2008. The finalists were announced on April 16. I would like to write down how this contest is organized. This post can be interesting to the contest participants and the reviewers.

Organization

This is the third episode of this contest. As service computing and Web services courses are taught in more and more universities, the submissions and the quality of the submissions are keeping on increasing.

This year, most submissions are from Canada, China, Germany and US. Maybe my connections to Canadian and Chinese universities play a role in inviting the submissions. I am especially glad to see that the Chinese universities are eager to use the latest techniques in their projects. The improvement of the research facilitates and investment is tremendous in China. US has considerably low submissions. If considering the adaptation of SOA in their market, this number is not compatible. We should contact more US universities. Well, many things are determined by the education and research systems. Germany is very active in the past competition. And they submit high quality papers.

This contest is designed to promote teaching in universities. So we expect to see projects at the scale of a course project with multiple students at graduate and undergraduate level. And it is what I saw – some professors send more than one teams to the contest from the class. I saw teams from semantic Web, Internet programming, or service computing courses from several universities. While some of the submissions actually are from the results of national programs. For example, I saw projects for public transportation systems and national wide database queries. These projects are normally funded by national programs. It is both good and bad. But so far, it does not need to say unfair to compete with these projects. It is obvious that budget and the size of the team are not correlated to the quality of work. If things change, we will re-evaluate the situation.

Evaluations

We use double blind review. We have ranked 16 submissions. We accepted 13 papers to publish and 11 teams are invited to the final. I would say that the top runners demonstrate high quality of work, from technique depth and the quality of the demonstration. The ranking of the review is not announced. The finalists will compete for 6 prizes at the conference venues.

The final contest

The final contest will be held in both ICWS and SCC. This contest is organized in two conferences is because some students had trouble to get US visa in the past contests. We will keep the evaluation the same in the two conferences. As you can see in the Web site, the results will be announced in ICWS. That means some of the winners may not receive their prize by themselves. So actually attending the contest in Beijing has some advantages, especially for the teams who will receive the prize. So far, the teams are very even in the two conferences - 6 teams in Beijing and 5 teams in Hawaii. And each venue has teams from all countries as well. I would say it is almost a prefect uniform distribution.

In the online review stage, the demo system is just one of many criteria. It weakens the weight of the demo system in the evaluation. Some demo systems are not so complete as their papers suggested. I would suggest the final teams to put a lot of efforts to enhance the demo system, because this is going to be weighted a lot in the final.