ViSenze Blog

Demystifying Machine Learning Model Development

Posted by ViSenze 29-August-2018

Do you want to train your own model on a customised AI model rather than starting from scratch?


As Artificial Intelligence usage increases across industries from security to healthcare, there is a matching upsurge in novel use cases that traditional out-of-box solutions can no longer answer to.

Recognising pastries themselves instead of barcodes for a self-checkout system at a bakery? Built-in image recognition for fridge cameras to identify expired foods? How about training models to analyse medical imaging for disease diagnosis?

While standard solution offerings may cater to a wide range of commercial verticals, often they are unable to serve use cases as unique as these. In such cases, rather than forcing a puzzle piece that doesn’t fit, it is more ideal to acquire a customised AI model, or better yet, to build your own model with a training system tailored to your data set.



Screen Shot 2018-08-29 at 10.56.22 AM


Common misconceptions about Learning process


For those removed from the actual process, there often exists the misconception that the training of an AI model depends exclusively on the Machine Learning code (the likes of TensorFlow, Caffe, MXNet...). However, this training process actually comprises of a whole system with many sub-components that can individually affect the development of the final model. In reality, the ML code constitutes but a very small part of the whole system.

Screen Shot 2018-08-29 at 10.57.46 AM

Screen Shot 2018-08-29 at 11.12.51 AM


As shown in the figure above, aside from the ML code, the requisite infrastructure and tools within the model training system are intricate, expansive and equally as important as the code itself. The diagram below shifts the traditional conversation’s heavy emphasis of ML code to the other component parts by illustrating their respective roles in the model training process:

Screen Shot 2018-08-29 at 10.59.00 AM


Time to rethink the ML model training process


There are two initial elements required for model development to begin. The first element consists of the training datasets derived from raw data that undergo data analysis and labelling. The second element is comprised of pre-trained AI models developed by a combination of data scientists’ expertise and infrastructural tools. The pre-trained AI models serve as a basic framework, the accuracy of which is continually improved upon by the training datasets to produce a specialised AI model; this process is captured within the ‘Model Training Engine’. Once the AI model is produced, it can start to analyse input data with machine learning algorithms to find patterns and synthesise predictions. If the accuracy of the produced model is found to be insufficient, then it may re-enter the model training engine and be further enhanced by the training datasets.


What are the real pain points in the process?


The multitude of variables within the whole model training system translates into a corresponding number of potential friction points that may emerge when resource deficiency occurs. The two most significant prospective issues stem from the two initial elements, with the first being the labelling of raw data.

For image recognition models, raw data takes the form of image data, which are more difficult to label than structural data. However, businesses often underestimate the impact of this seemingly insignificant phase in the model training process and consequently lack the mechanisms for effective labelling. Correctly annotating the raw data can greatly improve the compatibility of the resultant training dataset and in turn increase the efficiency as well as the scalability of the model training system as a whole.


Pain Points


However, the data labelling process is carried out by the manual efforts of data scientists, who on average dedicate around 70% of their time working on the entire system to the process alone. This leads to the other major latent pain point at the core of AI expertise: the recruitment of data scientists for such expertise is a difficult and expensive process. Given the extensive qualifications needed for the role, the workforce numbers within the profession cannot keep up with the rapidly growing demand of the industry. Data scientist roles often require around three to five years more experience than other technology roles, and as a result, the demand for data scientists specifically has risen to 50% higher than that for all other technology professionals; this demand is predicted to grow by another 15.4% by the year 2020.

Moreover, the steep ascent of the hiring process doesn’t stop here; companies must continually work to retain data science talents and ensure a high job satisfaction rate, since they are often unwilling to perform menial tasks, such as data cleaning, that don’t maximize their utility in their service to the enterprise.2


Although these two friction points may appear inconsequential due to the disproportionate emphasis placed on the ML code, they reserve the potential to undermine the efficacy of the entire model training process. To address these friction points and minimize their impact on the output of model development, ViSenze is launching, a machine-learning platform for AI model development. Principled upon collaboration and annotation, this ML training platform accelerates the idea-to-product cycle for businesses and includes features that minimize the resultant impact from these friction points


Contact us if you want to know more about this space.



  Industry news,   E-commerce,   New product features,   Deep learning