top of page
Mauricio I. González Evans

Orchestrating different FactSet data sources into one cache. A case example.


Or: How to get all fields from different sources into a single cache record while ensuring real-time fields are always up to date for snapshots and entitled correctly.

FactSet provides a large set of WebAPIs, SFTP downloads but also real-time APIs for accessing quality market and associated data as well as historical data, reports and much more.

Today customers do not want to deal with the ever-increasing number of APIs and streaming data delivery technologies. Instead, a simple, single point of access for market data, without having to care what kind of technology delivers the data, is in demand.

To comply with this requirement the complete set of data available for a symbol needs to be prepared and made available over high-speed delivery mechanisms ideally in the cloud i.e., a Multi-Source Integrating Cache Array: MuSICA.

The ONE MuSICA (Multi Source Integrating Cache Array)

  • is the single point of access for all most accurately updated data belonging to one symbol (master key)

  • provides access either on-prem or out of the cloud or both

  • delivers a single API for request-response and real-time streaming data

  • is virtually endlessly scalable by increasing the amount of caching microservices

  • supports thousands of simultaneous cache sources

  • lightning-fast cache updating and record delivery (< 0.01 ms latency for > 1Mio requests)

  • 500k+ cache responses/second with 250+ fields

  • 10k+ simultaneous consumers

  • fully permissioned cache filling and data delivery (entitlement enforcement)

  • cache arrays are fully resilient and failover time is <1s

  • all connectivity and low latency data transfer over Solace middleware (VMR)

For a specific customer use case, the following FactSet data from different sources needed to be consolidated into one record:

  • FactSet broadcast API for real-time streaming market data (incl. OPRA)

  • Meta data (e.g., Master Security Data), provided via cron every morning at a specific point in time

  • Reference data via WEB-API, requested by cache fillers running early in the morning

  • Different WebAPI sources for data not available over broadcast API

  • Real-time calculated indicators

Result:

  • No customer side installation required (AWS/Docker); customer only provides connectivity to the cloud VPCs

  • The MuSICA cache from this use case consolidates five different sources that would have required four different access technologies

  • data delivery in lightning speed, fully entitled

  • very simple adaptation of existing applications by using the ONE API for any data (Java/C/C++/.Net)

  • entitlement for multi-user services using openMECS

All MuSICA data is also available in Excel using snapshots and/or real-time subscriptions.

Author: Mauricio I. González Evans

Contact: mge@bccgi.com


46 views

Recent Posts

See All
bottom of page