As you might be aware I have a degree in Neural Networks (e.g. Neural Networks, AI and ML). During obtaining of that I had few concerns and ideas, as you can see those concerns of mine led us to another discourse article. Don’t mind to skip it if it is not why you’re on Simplex Code.
From where I stand it looks like building an intuitive and progressive website using cutting edge technologies is the primary goal of any web developer. They need to go through several processes, including building prototypes and testing the website before pushing it live.
Not a secret that these processes don’t happen overnight. In fact, developers spend months to build a beautiful and responsive website. That why any software engineer is obsessed with advancement in technologies, which is making things easier for them.
Modern technologies like machine learning and artificial intelligence are accelerating front-end development and making coding and testing of web layout much more easier, efficient and the last but not least faster. So lets figure out how!
Training Neural Networks
Stick and stones may break my bones but I am not considering Deep learning as a sub-category of Machine Learning, moreover I insist that there is no any difference in those notions…but probably I should live it for another article. So in general machine learning(ML) involves training and inference of machine (no way!). Obviously if there are artificial neural networks than they need to be trained. Users will need to feed datasets to an untrained neural network along with the right answers (so called confirmation). What this means is that training will include a large amount of input and respective output.
Following the training, the neural network will become capable to analyze and understand the input which is alike to inputs that network was trained on and then return the output accordingly. Primarily, the deep learning models are used for several use cases of AI, for example natural a language processing or a machine vision.
So relying on fact that neural network may reach anything based on the right input - it can be argued that Front-End developers or Web designers can use similar models for designing and creating UI elements only by training suitable neural networks with relevant data.
Machine Learning in Front-end Development
In my opinion so far there are only two types of data that can be used to train a neural network in UI purposes. The first one is a graphical user interface (GUI) screengrabs and the second one is sketches. The screenshots and sketches would hold their respective code.
This training, that will make the neural network able to generate code from a GUI screenshots, is called pix2code. Likewise, the conversion of sketches to code is called sketch2code. After analyzing the screengrabs and sketches, the neural network will find out the way to code those images in HTML or CSS. For text parts the networks will have to go through text recognition steps, which are may be considered as a routine in neural network training.
Following these essential steps, the neural network comes into action and leverages its knowledge of objects, their positions and design layouts for generating code for design elements. Eventually, this neural network becomes more capable to understand visual design elements and the intention of coding easily.
Although the power consumption of deep learning is higher, the use cases of this technology remove the roadblocks in web designing and make the entire process seamless.
Since there are millions of websites out there, machine learning engineers don’t need to worry about the data needed as input for training deep learning models. Billions of websites are pushed live every single day, which means that the training can be done with up-to-date design elements.
Microsoft Sketch2Code Tool
Microsoft, the well known tech giant, is deeply involved in deep learning (ha, good one) and artificial intelligence. Apart from making world a better place this is leeds to appearance of variety of auxiliary software development tools. One of those tools the Sketch2Code tool by Microsoft AI Labs. Sketch2Code uses deep learning to create a system that can understand the things drawn by the designer on a whiteboard, then the system interpret the understanding into HTML code.
Here is a sample of how the service streamlines the process of UI design.
Microsoft has trained its Sketch2Code system with huge set of images. It can detect objects for a broad range of preseted. The process is simple: upload your hand-written design, Sketch2Code will be at work, review freshly-baked-AI’d-page and finally download it.
So Where Does It Apply
Large enterprise companies already have started to use AI-based Front-End technology at least to power their UX design.
Airbnb, the largest community-driven hospitality company in the world, is applying AI to create its own machine learning design tool which has it’s working title - The Prototype. Benjamin Wilkins, Airbnb design technology leader, says that the time required to test an idea should be equal to zero. With this goal, the company uses the Prototype to generate a workable design code from sketches.
Initially, Airbnb had trained the Prototype with only a dozen hand-drawn sketches, machine learning algorithms, as well as some intermediary code for rendering elements from the design system into a browser.
The other good example is Uizard. Uizard it is startup based in Copenhagen, uizard’s team use the concept of converting images into HTML make-up for their entire business. The startup has built a machine learning algorithm that can read images and return the production-ready and customizable code related to the uploaded platform.
Uizard decided not to stop on web development and applying this concept as well on the development of mobile applications.
What about UX Engineers and Front-End developers…Is it time to panic?
No worries mate! When artificial intelligence is used to power Front-End development, the designers and developers get more time to focus on creative tasks. You, guys, spend several hours and days finding out relevant graphic design or interpreting someones sketches.
Of course streamlined process of Front-End development will enhance the way enterprises work - isn’t it what we used to call progress. If the senior executives want to try out new cutting-edge technologies than sooner or later, the developers wouldn’t need to spend the same time again and again.
Furthermore, web developers and companies will be able to predict the time required for the production of a project. Eventually, the optimization of time, resources and design budgets can help companies to reduce costs and increase so called ROIs (returns on investment). And we all know how lower cost of job for same amount of profits leads to higher wage, what is quite a goof news…I guess.
I do believe that within the next couple of years, AI and Machine Learning will completely change the way software developers work on the Front-End of an any project. But hey, nothing scary here, I can recollect alike issue in the past - back in the 1964 was innovated first computer mouse.
From where I stand it dosen’t look like keyboard-making companies went bankrupt. Moreover, enterprises providing high quality peripheral equipment seems to be most successful. Same thing with AI on Front-End - it will speed up the prototyping and break the barriers of software development, not destruct you further career path.
Wrapping Up
Application of Machine Learning in Front-End development can revolutionize the way developers, designers and companies operate so far in internet space. What developers need to do is to become familiar with modern technologies so that they don’t get a deadlock stage, not because of AI on every corner, because it’s developer terminal duty. Duh!
Regardless the fact that both of my personalities (Web Developer Guy & AI Fellow) itching to have a fight - AI and Machine Learning wouldn’t replace developers, rather, these technologies will assist them in becoming more creative and allow them to focus on fundamental tasks, taking time consuming essential steps.