Joint team from Flytxt, IIT-Delhi and CEERI wins NeurIPS 2018 AutoML challenge
December 28, 2018, Mumbai
Flytxt, a market leading supplier of analytics and Artificial Intelligence based customer value management solutions informed that the joint team named “autodidact.ai” from Flytxt, IIT-Delhi and CSIR-CEERI was awarded first place in NeurIPS 2018 AutoML3 Challenge, during NeurIPS conference which recently got over in Montreal, Canada. The challenge was jointly organized by 4Paradigm , ChaLearn, Microsoft and Arcadia University. The AutoML challenge is in its third year and this year’s task was to design a computer program capable of developing predictive models without any manual intervention that are trained and evaluated in a lifelong machine learning setting.
Several real-world machine learning applications extensively rely on the expertise of data scientists and domain specialists to solve, however, this is not a scalable solution. AutoML is an emerging AI discipline which attempts to automate the end-to-end process of applying machine learning to real-world problems. This presents continuous learning, or Lifelong Machine Learning challenge for developing an AutoML system, and that was the central theme of the NeurIPS 2018 AutoML challenge.
Around 300 teams from top universities and Industry including MIT, UC Berkeley, Peking University, Microsoft, Alibaba, and Tencent participated in this challenge. It was conducted as part of the Neural Information Processing Systems i.e. ‘NeurIPS 2018’, one of the most prestigious conferences on Machine Learning and computational neuroscience. The event also enjoys the distinction of being the world’s largest AI academic event with over 9000 delegates attending it this year.
The challenge spanned over 2 phases, namely Feedback phase and AutoML phase. Feedback phase allowed participants to practice on 5 public datasets. AutoML phase was a blind phase with no code submission. The last submission in the Feedback phase was automatically trained and tested, without human intervention, on 5 new (and undisclosed) datasets and the aggregated results (based on AUC metric) on these 5 new datasets was considered to determine the winners.
The joint winning team from Flytxt, IIT-Delhi and CSIR-CEERI developed an AutoML framework named AutoGBT (‘Automatically Optimized Gradient Boosting Trees’). AutoGBT essentially involves an adaptive self-optimized end-to-end ML pipeline consisting of a stream processor and a frequency encoder to exploit semantic similarity of categorical and multi-valued categorical feature values across batches. The adaptive model pipeline performed well in the blind phase by utilizing a Gradient Boosting Trees based learning algorithm along with automatic hyper-parameter tuning using Sequential Model-Based Optimization. A combination of scientific principles and clever engineering helped the team to withstand the challenging test in blind phase. AutoGBT is now open sourced and can be accessed from our GitHub repository.
The second place was secured by the team from Tsinghua University in China, whereas the third prize went to the team from Beijing University of Post and Telecom and Central South University, China.
