A TikTok Shop API for public product data: price, units sold and stock per SKU as JSON
Disclosure: I build TubeText . This post uses our TikTok Shop Actor on Apify. The numbers below are from a run I did today. If you search for a "Tik…
DevOps
⚑ Report a ProblemLatest DevOps news from Tech News
All topics
agents
ai
api
architecture
automation
aws
backend
beginners
cloud
database
devchallenge
devops
docker
javascript
kubernetes
llm
machinelearning
mcp
node
opensource
performance
productivity
programming
python
security
showdev
testing
tutorial
typescript
webdev
Track new job postings from Naukri, StepStone and Job Bank Canada for a niche role (Python)
Disclosure: I built the Apify Actor used in this tutorial. This post was written with AI assistance. If you are watching the market for one specific…
How to Compare More Than 5 Keywords in Google Trends on One 0-100 Scale
If you've ever tried to chart 12 keywords from Google Trends, you've hit the wall: the compare box takes 5 terms max . The obvious workaround (run t…
8% of My Capterra Reviews Arrived as the Three Characters $62
I run a visa agency in Bali. The half of my week that is not visas goes into Apify Actors, and one of them reads B2B software reviews — Capterra, So…
JobSpy doesn't scrape company career sites. Add Greenhouse, Lever and Workday jobs in the same DataFrame
JobSpy (speedyapply/JobSpy on GitHub) is a popular Python library for job search data (about 4,500 GitHub stars): one scrape_jobs() call returns Lin…
Why your Instagram scraper works on your laptop and dies on the server
The most common message in our support inbox is a variation of “it worked yesterday on my machine”. It did, and it will again — on that machine. Ins…
Booking.com Search Rows Hide 9–16% of the Price in a Field Labelled 21, 22 and 23
I run a visa agency in Bali. The other half of my week goes into scrapers on Apify, and one of them reads Booking.com search pages: destination in,…
Your Scraper's "Success" Metric Is Lying When the Site Sends You an Empty 200
Most collection pipelines count a fetch as a success when the HTTP status starts with 2. That works right up until the day a site hands you a polite…
How to Scrape Real Estate & Car Listings from OLX Brazil with OLX Brazil Scraper
1. Introduction Extracting market intelligence from Latin America's largest classifieds marketplace— OLX Brazil ( olx.com.br ) —can be a technical n…
How to bulk-audit local business websites
If you sell local SEO, web design or lead-gen services, you probably look at a lot of small-business homepages. Checking them one at a time in a bro…
Why youtube-transcript-api gets blocked on your server (and 3 ways to fix it)
If you've used youtube-transcript-api , you know the feeling: it works perfectly on your laptop, you deploy it to a server, and every request fails…
Scrapy vs BeautifulSoup for Public-Data Collection: Where Each One Starts Costing You
Scrapy vs BeautifulSoup for Public-Data Collection: Where Each One Starts Costing You Every scraping tutorial starts with pip install scrapy or pip…
How to track new jobs from Greenhouse, Lever and Ashby (daily alerts, no HTML scraping)
Many startups and tech companies host their careers pages on Greenhouse, Lever or Ashby . All three publish a free, public job-board API : the JSON…
Proxy format: IP, port, username and password, every form
Originally published on proxymint . Read and write proxy strings: host:port:user:pass, user:pass@host:port, http:// vs socks5h:// URLs, separate fie…
SOCKS5 vs HTTP proxy: DNS, speed, security, which to use
Originally published on proxymint . SOCKS5 vs HTTP proxy compared: what each carries, where DNS resolves (socks5 vs socks5h), speed, auth, client su…
Cloudflare Is Changing How AI Agents Access the Web in 2026
AI agents are getting better at using the web. They can search for information, read documentation, compare products, research topics, and collect d…
How to Scrape Google Jobs with Python (Salaries, Apply Links and Daily Alerts)
Google Jobs is the job search built into Google. Search for "data analyst Chicago" and Google shows a box of listings pulled from LinkedIn, Indeed,…
Check a hotel's price on every booking site (and spot rate-parity gaps) without writing a scraper
Google Hotels quietly does something very useful: for any hotel and any dates, it lists the price on the hotel's own website and on Booking.com, Exp…
We configured residential proxies on our MCP server. They did nothing. Six releases later.
"What browser MCP to bypass Cloudflare?" is a live question on r/ClaudeAI, and the honest answer for most of 2026 was: none of them reliably, ours i…
The wrapper that made my overnight AI scraper safe to leave running
I wanted a small research database to grow from one subreddit to ten communities where people talk about building with Claude Code and automation. D…
How to track Google Hotels prices with Python (no browser, no API key)
I built this actor; it's a paid tool on Apify with a free trial credit. Google Hotels shows you a price for every hotel in a city, but there's no of…
Getting YouTube transcripts without the Data API — and why "translate to 100+ languages" is usually broken
The YouTube Data API does not return transcripts. captions.download exists, but it only works for videos you own . For anybody else's video the offi…
Building a Serverless AI Tech-News Bot with Node.js, Gemini and GitHub Actions
I wanted to build a project that combined several technologies into one practical automation pipeline: Node.js Web scraping RSS Offline NLP Generati…
We timed HTTP-only scraping against a headless browser on the same page. It wasn't close.
Every one of our Actors is HTTP-only — no Puppeteer, no Playwright, no headless Chromium in production. We say this is faster and cheaper. This post…
What crt.sh's Error Pages Taught Me About Retry Logic
I built a small Apify actor that watches Certificate Transparency logs for a domain, basically "tell me every SSL cert that's been issued for this d…
When curl, TLS spoofing and headless all fail: how I got past Akamai with a real browser
I scrape data for a living, and most of that job is the same fight over and over: a site has data I need, and it really doesn't want me to have it.…
An isinstance(str) check that was always False capped every Facebook Page at exactly 3 posts
Quick answer: we asked Facebook's own internal pagination for 40 posts per Page. Across three unrelated Pages — NASA, Nike, Microsoft — we got back…
Scraping Adversarial Municipal Portals: A Permit Pipeline That Knows When It Failed
Most California cities publish their building permits, and almost none of them publish the same way. The San Francisco Peninsula alone runs six vend…
Turning Japanese press releases (PR TIMES) into an LLM-ready dataset in 5 minutes
If you build anything that needs to understand the Japanese market — competitor monitoring, PR analytics, a RAG assistant for a sales team — you qui…
A one-step probe told us YouTube was blocked. It was asking the wrong endpoint.
Quick answer: If you POST to YouTube's internal /youtubei/v1/next with just a videoId and count zero comments in the response, you have not found a…