Analytics Challenge - Submission, Judging, Awards and SC08 Activities

Note – this information will be updated with detailed schedule information early in 2008.

The Analytics Challenge encourages innovative and sophisticated analysis and visualization techniques in a meaningful application that supports the discovery of knowledge. The discovery of knowledge can be exemplified in a number of ways such as the identification of noteworthy findings in major research projects, real insight into key relationships between data in multivariate datasets, detection of distinct data characteristics, clear understanding of trends over time, the ability to model real-time scenarios with accuracy, dependable and meaningful forecasts, and more

Some of the key features important for applications to address in the Analytics Challenge include:

Innovative and sophisticated analysis and visualization techniques

Performance and scalability (with large, complex data sets)

Portability and distribute-ability

Flexibility and deploy-ability

Knowledge & discovery

Additionally, the applications will be judged on the strength of the solution meeting the combination of various specific criteria such as:

Utilizing various HPC technologies

Analysis: Data mining, forecasting and optimization techniques, etc.

Visualization: Interactivity of the application such as drill-down, real-time rendering and sophisticated visualization capabilities, etc.

High performance computing

Newer technologies such as web services, clusters, grids

High performance storage

High speed networking

Implementing unique investigation and methods for data analysis

Handling massive amounts of data

Providing for collaboration, sharing, and ease of use of the application

Achieving scalability and portability

Providing significant and meaningful results from the use of the application

Applications for consideration ideally would focus on solving real-world, complex problems in a number of sectors. Some examples of these areas are:

R&D/National Labs/ Homeland Defense, DOD, DOE (security, disaster planning)

Oil and Gas (oil drilling optimization)

Large Scale Science (astrophysics, High Energy physics, etc.)

Finance (risk management, portfolio optimization)

Corporate management (business analytics)

Life Sciences or Healthcare (clinical trials, pharma, bio-medical research)

Music and multimedia characterization and study

Environment (predicting weather, forecasting, remediation)

Web search engine enhancements

SPAM and Intrusion detection or mal-ware detection

Analytics Challenge - Submissions and Judging for Challenge

The Analytics Challenge is open to everyone except those directly associated with the Analytics Challenge Peer Review Committee. If in doubt, please make an inquiry to the Analytics Challenge Chair for the SC08 Challenge: Paul Fussell (paul.fussell@boeing.com).

You may use any existing commercial products and applications or research prototypes. And, you may use and combine whatever tools you deem necessary.

Submission Information: What to Submit?

Notice of intent to participate in the Analytics Challenge: due May, 2008
Register and submit your intent to participate to the Analytics Challenge section at the conference submissions website: www.sc-submissions.org
We use the Notice to Participate for planning the judging schedule as well as helping us choose judges with the right domain expertise. Your submission stating your intent to participate in the Analytics Challenge for SC08 would include the following:

Entry title for your application

Contact information

Student or professional entry

Your organization

Brief description of the challenge being explored, the dataset being used, the visualization you are using, and your hoped for solution

Team leader information, technical point of contact, and team members

Upload a PDF Extended Abstract describing your intended Challenge solution. This Extended Abstract should not exceed two pages in length. Please address the following:

A brief statement of purpose of your Challenge solution

A brief description of the methods to be used in your Challenge solution

A short listing of the major tools or codes to be used in your Challenge solution

A substantial discussion of the dataset or datasets used in your Challenge solution, including:

Overview of the data set

Dataset size

Dataset complexity

Dataset age: real-time, near real-time, or historical

How this dataset is to be used

An overview of methods to display results: Discussion of the methods, tools and/or codes used to display or visualize the results of the Challenge entry.

A discussion of the hoped for overall results of your solution’s operation

Specify the format of the visualization you’ll be presenting with your completed solution. We very strongly prefer videos playable using Quicktime version 7, or Windows Media Player version 10. DVD in DVD-R format is acceptable, but not preferred. Higher resolution is likewise important.

Completed Solution to the Analytics Challenge including results: due in July 2008
A Complete Solution comprises

A complete solution

A maximum 150-word summary for the SC08 program

A very capable video demonstration

Completed Solution

Upload a maximum six-page publication quality PDF summary that addresses each of the individual judging dimensions (analytics techniques, robustness, flexibility; performance and scalability; knowledge & discovery; results) while showing how the science is advanced by your work. To best support the process of judging your Solution, the summary should:

clearly identify the core problem being addressed

show a thorough overview of the data

describe the analysis process

give insight into the visualization challenge and your solution

summarize the strengths and weaknesses of the approach

provide at least one example screen shot

The summary should adhere to the following formatting guidelines:

The page limit includes images, tables, code snippets, etc.

Margins: 1-inch margin on all sides

Font-size: No smaller than 10 pts should be used

Spacing: Single line spacing for all sections

Video Demonstration

The visual video of your work has proven remarkably important in judging your entry. We ask for the recorded demonstration because interactivity is an important component for the Challenge and it is difficult to evaluate your work solely on considering a paper submission.

The video should be playable by either Quicktime version 7, or Windows Media Player version 10. Please produce the video at as high a resolution as practicable. Videos presented on a DVD in DVD-R format are acceptable. The recorded video demonstration should show the context of your work, and should focus on the interactive features of the completed entry and the process of analyzing and displaying the data. Mixing voice, pictures, screen activity, and possibly the real-world implications of your work is preferred. Demonstrations should be from five to ten minutes in duration, and will not be considered beyond twelve minutes duration.

The video shall be submitted in one of two ways: (1) provide your URL from where the video can be downloaded to the judge’s computers; (2) ship a CD authored by you containing the video as a file, to the Challenge Chair. The URL must be active through SC08 and preferably for several weeks following that date.

Analytics Challenge - How to Submit

The entry form and summary (in PDF format) are to be uploaded to the SC-Submissions web site (www.sc-submissions.org).

The video shall be submitted in one of two ways: (1) provide your URL from where the video can be downloaded to the judge’s computers; (2) ship a CD authored by you containing the video as a file, to the Challenge Co-Chair. The URL must be active through SC08, and preferably for several weeks following that date.

Analytics Challenge - Judging Criteria

Submissions will be subject to a peer review by judges with analytics expertise. All submissions will be considered with the following three major dimensions in mind:

Analytics Techniques / Robustness / Flexibility

Flexibility of the application (how generic is your approach)

Quality of the written case study (description of the strengths AND weaknesses of the analytics used)

Novelty (whether the analysis and visualization methods are common approaches, well known versus more innovative and sophisticated approaches)

Performance & Scalability

Performance of the application considering the data (would the application struggle if the data set grew to a larger size or greater complexity)

Capability of the method to scale with processors and network (is the method linear, or superlinear, polynomial, exponential, with time; inversely with processors and network speed?)

Quality of the architectural components used to solve the problem

Complexity of the data

Knowledge & Discovery (Results)

Are the results interesting (given the problem)

Are the results impactful (are they important)

Quality of the insights that were achieved

Usefulness of the application (does it produce creative outcomes)

Quality/sophistication/appropriateness of the visualization techniques to illustrate results

Verification of the results (are they qualified)

Analytics Challenge - Awards

Winning entries will receive a special certificate and a nominal monetary prize and will present their work during the contest session at the conference. The length of the oral presentation is still to be determined but is likely to be similar to a short paper or presentation as part of the Analytics Challenge session in the Technical Programs schedule. Also, plans and preparations will be undertaken such that winning entries would also show their applications "live" at the SC08 Conference.

We will award prizes for two categories based on the material received - "Best Overall" and "Best Student Entry".

Honorable Mentions will be forwarded on to the Technical Programs Committee for consideration for being presented as interactive posters at the conference.

Important Note: All award winners are required to attend SC08 and participate in the Analytics Challenge activities.

Dear Colleague,

Ann Redelfs

2007-11-14

Christopher P. Hamb

Cornell University