No-API external retrieval automation

Some source documents didn't live behind an API. They lived behind a login screen, a portal maintained by a bank, fund administrator, or counterparty, with no programmatic access and no intention of building any. The industry standard was a vetted human holding standing credentials, checking in on a schedule, and downloading whatever had been posted.

That approach had a cost, but it wasn't primarily speed. Manual checks were already fast enough, next business day was the norm. The actual barrier was trust. Extending and paying for standing portal credentials to several members of another party, even a vendor a client already worked with, was something institutional clients were reasonably hesitant to do. Every portal was a small, standing grant of access that had to be justified and monitored.

• Portal retrieval. Browser automation that logs into a target portal, navigates it, and downloads newly posted documents on a continuous schedule, without a human holding the credentials day to day.
• Automated navigation and login, including handling two-factor codes delivered by email, so retrieval didn't require a human in the loop for routine access. This used an existing system for processing documents, so reading the initial notification email that kicks off the process and the 2FA email were already solved.
• Failure detection, using a visual model to recognize when a portal's layout had changed or a step didn't produce what was expected, rather than silently returning nothing or the wrong thing.
• A contracted fallback. When automation failed, an approved third party was notified to retrieve the document manually, with an obligation to respond and staff the retrieval.
• Reconciliation logging, so every download was checked against what the client's system expected, closing the loop between what a portal published and what was actually ingested.

An integration is commonly thought of as an API connection, an established handshake between two systems, and a recognized event structure.

There's no API for a login screen. So the interface became the integration.

Browser automation against a portal with no API and no version guarantees can inherently be fragile. A layout change, a new consent screen, a moved button, can break a script that worked yesterday. That fragility wasn't something to solve, it was something to bound.

The system never had to be perfect, it had to reliably know when it wasn't working, and hand off cleanly when it didn't. In practice, the automated path held and the fallback was rarely needed, but the guarantee is what made clients comfortable enough to adopt it in the first place.

This product removed a significant adoption barrier, as it was often the highest touchpoint since documents either had to be manually downloaded and uploaded by the client or a third party had to be approved and provisioned access.

The principles used to build this product apply to any document, chart, or data that is locked behind a login screen. I used the same principles in my Stock Indicator Daily Check project, where the visual stock indicator chart surfaces insights to the user that retrieved computed results might fall short. This meant developing a way to access my account where the charts and their parameters lived, and navigating to retrieve and validate a screenshot for the LLM visual model to interpret.