2020年6月23日 星期二

Web crawling By Puppeteer @ Node.js - Part 2/2, How to make it work on Android?


Outlines

1. Install ubuntu on Android by AndroNix
2. Install node.js and puppeteer
3. start web crawling
4. advanced adjustments to display chinese characters


2020年6月22日 星期一

Web crawling By Puppeteer @ Node.js - Part 1/2

# Introduction


  • Puppeteer is a node.js library that achieves web crawling by an automated chromium browser (chrome open source project)
  • Unlike other methods needing to send pure http requests and parse responses yourself, even needing to handle cookies manually, your syntax is simply: 
  • Launch browser -> emulate real user operations on the web page -> get any data that is reachable by a real user

Web crawling by puppeteer is very simple.

This is part 1/2, in part 2/2 I'll demonstrate how to make it work on Android

# Getting Started

Start by installing:

  • npm install puppeteer