Content Fetcher Bot

The Content Fetcher bot pulls content and media into Tickaroo products. In a liveblog, it is used when you configure an integration:

  • RSS / Atom feeds — polling a feed you configure as a content stream.
  • Article / content import — fetching an article or page you import into a post.
  • Remote media — downloading images, video and other assets referenced in your content.
  • DAM / CMS integrations — fetching assets from connected systems such as CUE or Livingdocs.

In nxt, it is used for:

  • URL sources — fetching a page, feed or image a user adds as a source.
  • Reading pages for the assistant — fetching the text of a page the assistant is asked to work with.
  • Brand analysis — reading your homepage and a few of your articles to detect fonts, colors and tone of voice. See Brand analysis below.

User-Agent: Tickaroo-ContentFetcher/1.0 (+https://www.tickaroo.com/bot)

Firewall configuration

Allowlist these so feeds, imports, media downloads and nxt’s page reads work reliably. Tickaroo bots do not evaluate robots.txt.

If your site sends unknown visitors through a sign-in check, and that check exempts known bots by User-Agent, exempt the Content Fetcher from it too. Otherwise the Content Fetcher ends up in the check’s redirects, and nxt reports that your site sends it through a sign-in check it cannot pass.

Brand analysis

nxt’s brand analyzer reads your homepage as the Content Fetcher. To read your articles’ fonts and colors, it then renders a few of them in a headless Chromium browser:

  1. The browser first loads an article with Chromium’s standard User-Agent.
  2. If your bot protection (for example Cloudflare or DataDome) answers with a challenge or a block, the browser loads the article once more as the Content Fetcher: Tickaroo-ContentFetcher/1.0 (+https://www.tickaroo.com/bot), with the Sec-CH-UA client hint "Tickaroo-ContentFetcher";v="1".
  3. If that load is blocked too, the article is skipped. If your homepage or every article is blocked, the analysis stops, and nxt names what blocked it and asks you to allowlist the Content Fetcher.

The browser sends its requests from the same IP addresses as the Content Fetcher, which reverse-resolve to bot.tickaroo.com. Allowlisting by IP range or reverse DNS therefore admits the browser’s first load as well; allowlisting by User-Agent admits its second.

Opt-out

The Content Fetcher only retrieves sources you have configured. To stop it, remove the relevant integration in the Tickaroo editor — the RSS feed / content stream, the DAM or Livingdocs connection, or the import. In nxt, it fetches only the pages your users add or ask about, and the site a brand analysis is started for.