In this article, we will explain you how to easily implement your own website cloner with Node.js. So, it would be great if you could download a copy of the code and resources of the website to manipulate it locally right? Thanks to a pretty useful script that uses Puppeteer, this can be easily done within just minutes (and seconds after its implementation). This is only uncomfortable when we talk about code highlighting, as you can't compare the syntax highlighting offered by the browser with the one offered by your favorite IDE like Visual Studio Code, Netbeans etc. The easiest way to do this is by reading the source code of a web page, using Ctrl + U in Chrome for example, reading the JavaScript files as well as long as they're not minified (in websites that sell templates for example). log( "CHILD: url received from parent process", url) Ĭonst browser = await puppeteer.How many times, as frontend developers, we decided to implement a copy of some feature that a third party website has in our own website? I usually do this a lot, specially when there's no open source alternative for the feature and i don't want to write it from scratch because it would take a lot of time to end up with something decent. The code snippet below is a simple example of running parallel downloads with Puppeteer.Ĭonst downloadPath = path. 💡 If you are not familiar with how child process work in Node I highly encourage you to give this article a read. We can combine the child process module with our Puppeteer script and download files in parallel. Child process is how Node.js handles parallel programming.
We can fork multiple child_proces in Node. Our CPU cores can run multiple processes at the same time. 💡 Learn more about the single threaded architecture of node here Therefore if we have to download 10 files each 1 gigabyte in size and each requiring about 3 mins to download then with a single process we will have to wait for 10 x 3 = 30 minutes for the task to finish.
It can only execute one process at a time. You see Node.js in its core is a single-threaded system. However, if you have to download multiple large files things start to get complicated. In this next part, we will dive deep into some of the advanced concepts.
0 kommentar(er)
