What is the purpose of Challenge Data?

The Challenge Data was started in order to provide an easy access to supervised machine learning datasets for teaching purposes. The datasets are small enough so they can be studied on standard machines, and still retain the interesting features of real data.

Who is it open to?

The Challenge Data is primarily designed for machine learning students and professors, but registration is free and open to everyone.

What are the differences between the types of accounts?

Participant accounts are made for the actual competitors of the challenges. Students who participate to a challenge as part of a course project must choose this kind of account.

Professor accounts allow professors to create courses in which their students can enroll, track their students' performances and download their scientific reports easily. It is also possible to take part in challenges as simple competitors with these accounts.

Challenge provider accounts are made for our partners who provide the datasets. Please contact us if you would like to propose a challenge before creating such an account.

What is the academic ranking?

The academic ranking is based on the scores computed on the private half of the testing dataset. It is computed only once during the competition, using, for each participant who is active at the time of computation, the submission which gave the best results on the public half of the testing dataset. It therefore allows the professors and challenge providers to measure if overfitting occurred until the half of the competition. Another ranking based on private scores will be computed at the end of the competition.

How to signup?

Go to the sign up page and select the account type according to your needs.

Use a password with letters, numbers and special characters.

Check your mailbox (including your spam folder) for an email from Challenge Data, and click on the link contained in this email to activate your account.

If you are a Professor or Challenge provider account, a manual validation is still required to activate all functionalities.

As a participant, how do I register to a challenge?

Go to the Challenges page. Select the challenge you want to participate in.

If you participate as part of a specific course in an affiliated academic program, you must select the appropriate course in the list. Note that you cannot affect multiple challenges to a given course, so choose this challenge carefully.

If you want to team up with another participant, you may enter his or her email address to send an invitation.

As a participant, how do I submit my predictions?

Once you are registered to a challenge, you can submit a prediction file from the "Current ranking" tab.

The prediction file must be a .csv file with the correct number of lines and columns, otherwise the evaluation script will raise an error.

Along with the prediction file, you can enter the name of the algorithm you used and the description of the parameters. These pieces of information will remain invisible to the other participants, so do not hesitate to fill them precisely. They will be stored in your personal space, allowing you to track your progress on this challenge.

You cannot submit more than two files every twenty-four hours on each challenge.

Are there rewards for the participants?

In addition to the potential rewards offered by the challenge providers on each challenge, the Challenge Data organization will reward the best participants over the whole competition.

To enter this meta-challenge, it is necessary to participate in all the challenges offered during the season.

Participants which have achieved consistently very good scores in most of the challenges will be declared winners of this meta-challenge.

The winners will be offered a prize during the closing ceremony in June 2017.

What is the purpose of the report upload?

All participants are encouraged to write a report as a scientific article, analyzing and explaining the performance of their algorithms, and providing references of published articles used in their work.

As a professor, how do I create a course?

As a professor, once your account has been validated, you have access to a new tab in your personal space called My Courses. You can create a course by clicking on the link "Create a course"; you are then asked to enter the name of the course. After its creation, the course appears below the link "Create a course".

After the creation of the course, you need to select which projects you want to link with the course. To access the list of the challenges, go to the Challenges page. For each challenge you want to link, click on this challenge. Once you are on the page of the challenge, click on "Link a course" and select the relevant course. Only the challenges linked with your course will be available to your students.

All the challenges linked to your course are reported in your personal space, under the tab My Courses. If you ask students to upload reports related to their works, you will be able to download all reports in a single click in this section, with the link "Get all reports".

As a professor, how can I get the results of my students?

Once a challenge is linked to a course, two new tabs appear on the page of this challenge: "Current ranking" and "Participants information".

The tab "Current ranking" gives you access to the ranking of the competitors obtained on the public half of the testing dataset, along with their scores. It is visible by all participants.

The tab "Participants information" provides you with more specific information: how many times your students submitted new solutions, their scores, a contact link and a link to download individual reports.

To download all reports in one click, go to My Courses and click on the link "Get all reports".