1.0.1 • Published 4 years ago
user-request v1.0.1
user-request
Simulate a user requesting an HTML page
Perfect for web scraping, and allows retry attempts.
You're able to bypass CloudFlare protection by using user-request
.
Install
npm i user-request
Import
const get = require('user-request')
Or...
import * as get from 'user-request'
Instructions
Call the get
function with a URL to fetch that page.
const response = await get('https://memorize.ai')
Pass options to the get
function
const response = await get('https://memorize.ai', {
maxRetryAttempts: 20, // Optional, 10 by default
headers: { // Optional, you can override or add headers here
'Accept-Encoding': 'gzip'
}
})
Properties on the Response
object
data
: The page's HTML content.headers
: The response headers.status
: The status code.statusText
: The status text.retryAttempts
: The number of retry attempts this request took.didFail
: If the request failed by either returning a 404 or 500, or running out of retry attempts.