Skip to content
daggerokPublic

About

Global X ETF holdings-to-watchlist: static ./api/globalx feed generated by a Bun updater from globalxetfs.com Next.js catalog and fund pages, dated full-holdings CSV

Topics

Resources

Stars

0 stars

Watchers

0 watching

Forks

Latest commit

 

History

59 Commits

Folders and files

Repository files navigation

Global X

One of the app's features lets you select Global X ETFs in the Watchlist and aggregate their holdings to see how often each ticker appears across the selected funds. Repeated holdings make overlapping exposure visible: the more selected funds include a ticker, the greater its potential influence on the portfolio; gains in that holding may help, while declines may hurt, and actual impact also depends on each fund's position size. Another feature makes it faster and easier to find funds with stronger growth over different periods, higher dividend yields or distributions, greater Total Return (price performance plus dividends), and other key performance metrics. A single-file client-side tool reading the generated ./api/globalx static feed (Global X ETF catalog and fund pages, dated official daily holdings CSV files, SEC EDGAR N-PORT-P and Yahoo Finance history) into a searchable ETF/asset-class catalog with per-fund tabs, watchlist aggregation, ticker copy and CSV/TXT export — the same look, feel, columns and business logic as the sibling applications.

Using Bun

bunx degit daggerok/Global-X#main ./12345 && cd ./12345 && bun install && bun run serve
open http://localhost:1234

The published application is available at https://daggerok.github.io/Global-X/. bun run serve starts the Parcel dev server and copies api/ to dist/api; bun run build builds the site into dist, and bun run build-github-pages does the same with the /Global-X/ public URL used by the GitHub Pages workflow (.github/workflows/github-pages.yml). The app reads only relative ./api/globalx/** files generated by scripts/update-data.ts; the committed feed is complete, so the built site needs no update run at all.

Column types and filters

Every column of the ETF catalog and of the Watchlist, Holdings, History and Distributions tabs has a type: text (ABC), number (123), percentage (%), money ($), date (D), date and time (DT) or time of day (T). The type is detected from the texts the column shows (80% of the filled cells must agree, otherwise text) and is written in the badge next to the column title: click it to cycle the type, Shift+click to return to auto-detection. Dates are read as 2024-06-15, 6/15/2024, 15.06.2024, Jun 15, 2024 or 15-Jun-2024, date and time as 2024-06-15T09:30:00Z or 2024-06-15 09:30, time as 09:30, 16:00:00 or 9:30 PM

A row of filter inputs sits under the column headers (the Filters button hides it, Clear filters empties it). Filters of different columns are combined with AND, the search box applies on top, and Copy Tickers and the exports use the filtered rows. Filters and type overrides are remembered in the browser. Sticky # (next to Filters, off by default, remembered in the browser) numbers the rows by their rank in the table sorted by the current column before the column filters, so a filtered fund keeps its rank and the numbers keep gaps; the sort, the search and the category and blacklist choices rank again. The catalog starts sorted by Net Assets, largest first, unavailable values sort last in both directions, and every export starts with the # column. The red Clear button opens a dialog that lists what can be reset (the selection, searches, sort order, open tab, shown columns, column filters, remembered table views and the blacklist), all ticked the first time and afterwards as they were left at the last OK; Enter confirms, Esc or a click outside cancels, the theme is always kept, so the page looks like a first visit (also after a reload)

Inside one filter: a space means AND, a comma means OR, a leading ! means NOT, ? matches an empty or unavailable value and !? a value that is there; a value that is unavailable matches only ? and negated conditions. An unquoted space ends the value, so quote values that contain one (>="2024-06-15 09:30")

Type Examples
Text bank contains, "two words", !bank, =exact, ^starts, ends$, /regex/, tech, health
Number, percentage, money >10, >=10 <50, =22 (matches what rounds to 22), !=22, 10..50, ..50, 10.., >1B and K M B T suffixes, an optional $ or %
Date, date and time >2024-06-01, 2024 (the whole year), 2024-06 (the whole month), 2024-01..2024-06, today, yesterday, -7d.. (the last 7 days), +2w, -3m, -1y
Time >09:30, 09:30..16:00, =12:00 (the whole minute)

The Columns menu next to Filters lists every column of the ETF table from the first to the last, all of them shown by default, with a search box and the All, Clear, Toggle and Reset buttons. Use and Ticker are listed but locked. Hiding a column only removes it from the table: the filters, the sorting, the exports and Copy Tickers still use it. The choice is remembered in the browser (localStorage, never the data) and the menu is shown on the ETF catalog only

The asset classes are one Asset classes multi-select next to the All ETFs pill instead of one tab per class: every class is selected by default (= all ETFs), Only or unchecking narrows the table, and the All ETFs pill is lit only while nothing narrows it (all or none of the classes checked); clicking the pill clears the selection. The choice is remembered in the browser (localStorage, never the data)

Updating the static Global X data

Run the updater with Bun:

bun install --frozen-lockfile
bun test
bun ./scripts/update-data.ts

Run bun ./scripts/update-data.ts -h (or --help) to print every control with its default and usage examples.

Defaults live in scripts/update-data.config.json, a flat JSON object that lists every supported control as a string. Precedence: file defaults < advanced JSON < nonblank workflow inputs < protected Actions variable/env. A blank workflow input inherits the file value, and advanced may set a key to an empty string on purpose. Unknown keys, non-scalar values and values with line breaks are rejected, and the CLI and the workflow use the same resolveControls.

The Update Global X ETF data GitHub Actions workflow (.github/workflows/update-data.yml) exposes the main controls as manual inputs plus an advanced JSON object for any other control, runs weekly on Sunday at 00:00 UTC, and commits api/globalx and nothing else. The protected SEC_UA repository variable wins over the file and inputs when it is nonblank. All supplied filters use AND logic.

Data sources

Block Source Notes
Fund universe, ticker, fund name, inception date, NAV, net assets, gross/net expense ratio, THEME and SUB_THEME globalxetfs.com/explore The current Next.js page publishes the catalog in its server-delivered React Flight payload. The updater decodes that payload without a headless browser.
CUSIP, ISIN, NAV, market price, daily changes, premium/discount, 30-Day Median Bid-Ask Spread, primary exchange and shares outstanding https://www.globalxetfs.com/funds/<ticker>/ The official fund page's React Flight payload plus its rendered ETF Prices table.
Holdings (daily) https://assets.globalxetfs.com/funds/holdings/<ticker>_full-holdings_<YYYYMMDD>.csv The dated Full Holdings (.csv) link on each fund page. The seven-column file publishes weight, ticker, name, SEDOL, market price, shares held and market value.
NAV-basis performance Official fund-page React Flight payload (PERFORMANCE_HISTORY) Monthly and quarter-end annualized and cumulative NAV values are copied from the provider payload. The catalog stores published annualized values and the app renders the same return rows as the sibling applications.
Distribution history and frequency Official fund-page React Flight payload (DISTRIBUTION_HISTORY) Ex-date, record date, payable date and amount are kept in newest-first order. Global X does not publish a declaration date in this payload, so that shared row stays blank.
Daily price history (Date / Close / Adj Close / Volume) Yahoo Finance chart API Used for the paginated history sheets and as the market-price fallback. Adjusted close is rounded to two decimals before writing.
Holdings fallback SEC EDGAR Form N-PORT-P, Global X Funds CIK 0001432353, file 811-22209 Used only when the official holdings CSV is unavailable and EDGAR_FALLBACK is enabled; off by default.
Dividend-history fallback Yahoo Finance dividend events Used only when Global X publishes no distribution-history rows; the feed marks those rows distributions.fallback: true.

The official Global X page was live-verified as a server-rendered Next.js shell with real catalog/fund values in React Flight payloads. No WAF proxy is used. Requests are bounded, retried, and paced by the configured per-worker lanes.

Metrics and caveats

The daily holdings CSV

The CSV is parsed with a dependency-free RFC-4180 reader that tolerates the BOM, CRLF and quoted fields. Global X publishes SEDOL for holdings rather than a separate asset-class column, so Asset Category is derived conservatively from the row's own name/ticker: cash/payables, Treasury, fund, option-like and otherwise equity. Provider values are otherwise preserved in the generated sheets.

Return arithmetic

The source publishes annualized NAV values for 1, 3, 5 and, where available, 10 years (TEN_YEAR) plus since inception, and cumulative monthly/quarterly readings. The feed keeps both readings and does not extrapolate a missing tenor; unavailable values remain null and the app renders —. A horizon longer than the fund's age at the performance date is null, and siAnn needs at least one year of history.

Returns provenance fields

Every funds[].metrics object in api/globalx/index.json (and in each meta.json) ends with two mandatory fields:

  • returnsBasis - non-empty text saying how the returns were computed. Global X returns are the official NAV total returns from the fund page NAV Performance table (monthly series), so the label is official Global X fund page NAV Performance (monthly series). A fund with no published performance yet carries an explicit none: ... label with every return null, never an empty string or -
  • performanceAsOf - ISO YYYY-MM-DD month-end date of that performance table (for example 2026-08-31). It is not the NAV date (asOfDate), which is newer. null only when the table date is unknown

Unavailable numeric values are null, never 0. Global X prints a distribution rate of 0 for funds that have not paid anything yet (including monthly bond funds with a positive SEC yield), so dividendYield and distributionYield publish null instead of 0; the 30-day SEC yield is published as the provider prints it.

Dividend yield basis

Every funds[].metrics object also carries dividendYieldBasis, a short code for the definition behind dividendYield. It is null exactly when dividendYield is null and always travels with the yield it describes (a retained fund keeps both from the same published state).

Code Meaning for Global X
official-distribution-rate The only code Global X rows use: the official fund page Distribution Rate (latest distribution annualized over the ex-date NAV)
official-trailing-12m, official-other, computed-trailing-12m, indicated Part of the shared vocabulary, never produced by this feed

Publishing rules

  • Each fund is computed completely in memory and published as a whole: page files first, then meta.json, then the index row, and stale pages are removed only after the new meta.json is written. A fund page without fund details, or any other failure, keeps the previously published state of that fund
  • A failed or skipped Yahoo read keeps the published history instead of publishing an empty one; the EDGAR fallback only accepts a filing of the same series and never replaces fresher holdings
  • JSON is written through a temp file and a rename. generatedAt and update-state.json move only when content moved, so a rerun with identical upstream data is a zero-byte diff
  • A row without funds/<T>/meta.json has dataFile: null and a full metrics object (all numbers null, non-empty returnsBasis)
  • terValue is the net expense ratio (the lineup NET_EXP, else GROSS_EXP), terGrossValue is the published gross ratio (GROSS_EXP)
  • Scheduled future distribution rows with an empty amount are dropped; a distribution rate of 0 is null
  • Funds new in the lineup are printed as NEW FUNDS: A, B and appended to the workflow step summary
  • Every request attempt has a 45 second deadline that also covers reading the body; the run stops taking new funds after 25 minutes and still writes the index

Known value limitations

Metric Status Reason
Declaration Date always blank in Global X distribution rows The current official DISTRIBUTION_HISTORY payload publishes ex-date, record date, payable date and amount, but no declaration-date field.
Underlying Exposure — It is not part of the current Global X catalog/fund-page payload used by the updater; the updater does not infer an index from a fund name.
SEDOL / CUSIP both may appear, in different places The daily holdings CSV publishes SEDOL; the fund page's Trading Details panel publishes CUSIP and ISIN. The shared Identifier holdings field uses the provider's SEDOL.
Growth-of-$10,000 series and document links — where the current payload does not expose a stable value The updater does not invent a chart series or document URL from an unrelated page.
Premium / Discount official value preferred; derived quotient retained The page payload's current premium/discount value is kept alongside (Market Price - NAV) / NAV as a guard.
EDGAR fallback opt-in SEC EDGAR can reject GitHub-hosted runner IPs with HTTP 403; the normal source is the official dated holdings CSV.

Update controls

Environment variable Default Meaning
MAX_FETCHES 0 Funds to process; 0 means a full pass. A positive value takes the next funds of the filtered set after the saved cursor in api/globalx/update-state.json and wraps around at the end. The cursor counts funds that pass the lineup filters (ticker, category, AUM, TER); yield and return filters are evaluated per fund after its page is read. A TICKERS run never reads, moves or deletes the cursor.
TICKERS "" Space/comma-separated ticker allowlist, combined with all other filters. A ticker the lineup does not list is an error.
CATEGORY "" Substring match on the official THEME / SUB_THEME category.
AUM / TER / DIVIDEND_YIELD / SEC_YIELD : Strict min:max ranges; AUM also accepts nano, micro, small, mid and large. DIVIDEND_YIELD and SEC_YIELD use the fund-page values, so a fund without that figure is excluded.
PERFORMANCE_YTD … PERFORMANCE_10Y / TOTAL_RETURN_YTD … TOTAL_RETURN_10Y : min:max filters on official/derived return values; a fund with no value for a bounded range is excluded, and an excluded fund keeps its published files.
CONCURRENCY / REQUEST_SLEEP 2 / 2 Per-worker request lanes and the minimum seconds between request starts within each lane; keep REQUEST_SLEEP at 2 or more, globalxetfs.com resets bursts.
HOLDINGS_PAGE_SIZE / HISTORY_PAGE_SIZE 250 / 1000 Rows per generated JSON page.
MAX_RETRIES 3 Retries after the initial request for network errors and HTTP 408/425/429/5xx; integer >= 1.
HISTORY_RANGE max Yahoo daily history window: max or <N>y, sent as an explicit period1/period2 (Yahoo ignores range when period1=0). Any other value is an error.
EDGAR_FALLBACK false Use SEC N-PORT-P if a holdings CSV is unavailable; off by default because EDGAR can reject GitHub-hosted runners.
SKIP_YAHOO / SKIP_GLOBALX false SKIP_YAHOO keeps the published history. SKIP_GLOBALX keeps every published figure and only refreshes the Yahoo history of already published funds.
STORE_RAW_DOWNLOADS false Keep raw catalog, fund pages and CSV samples under api/globalx/raw.
SEC_UA daggerok ETF feed [email protected] Declared User-Agent for SEC EDGAR, redacted in config logs; the protected SEC_UA Actions variable wins in the workflow.
VERBOSE false Print per-fund retry and fallback notices.
USE_SYSTEM_CA auto TLS trust store: auto restarts the updater once with Bun's --use-system-ca when a request fails with an untrusted-certificate error; true always uses the system CA store; false never restarts. Not an individual workflow input: use advanced, the config file or the CLI environment.

A bounded or partly failed run preserves previously published files. TICKERS combines with AUM/TER/yield/return filters using AND logic; it does not override them.

Config keys, CONTROL_NAMES, --help and this table are kept in sync by scripts/update-data.test.ts. HISTORICAL_PAGE_SIZE still works as an environment alias of HISTORY_PAGE_SIZE, and every control also accepts a GLOBALX_<NAME> environment variable that wins over the plain name (an explicitly empty value counts).

Examples

MAX_FETCHES=10 bun ./scripts/update-data.ts
TICKERS="PAVE AIQ COPX" bun ./scripts/update-data.ts
AUM="1B:" TER=":0.70" bun ./scripts/update-data.ts
CATEGORY="Fixed Income" bun ./scripts/update-data.ts

TypeScript and verification

The browser app lives in src/ (index.html, main.tsx, index.css with Tailwind v4) and is bundled by Parcel into dist - no tsconfig.json and no typescript package needed. Bun runs the TypeScript of the updater out of the box.

Verification before every publish: bun install --frozen-lockfile, bun test (updater tests plus the config, README and workflow parity checks), bun build --target=bun scripts/update-data.ts --outfile=/dev/null, and git diff --check.

Brands table

Brand Where to get the data
AAM aamlive.com | AAM
abrdn (Aberdeen) aberdeeninvestments.com | aberdeen
Amplify amplifyetfs.com | Amplify
ARK Invest ark-funds.com | ARK
Capital Group capitalgroup.com | Capital-Group
Fidelity fidelity.com | Fidelity
First Trust ftportfolios.com | First-Trust
Franklin Templeton franklintempleton.com | Franklin
Global X globalxetfs.com/explore | Global-X
Goldman Sachs am.gs.com | Goldman-Sachs
Invesco invesco.com | Invesco
iShares ishares.com | iShares
JPMorgan am.jpmorgan.com | JPMorgan
NEOS neosfunds.com | Neos
Northern Trust etfs.ntam.northerntrust.com | Northern-Trust
Pacer ETFs paceretfs.com | Pacer
Parametric eatonvance.com | Parametric
ProShares proshares.com | ProShares
Schwab schwabassetmanagement.com | Schwab
SP Funds sp-funds.com | SP-Funds
SPDR ssga.com | SPDR
Sprott ETFs sprottetfs.com | Sprott
Tema ETFs temaetfs.com | Tema
Themes ETFs themesetfs.com/etfs | Themes
VanEck vaneck.com | VanEck
Vanguard investor.vanguard.com | Vanguard
VictoryShares vcm.com VictoryShares ETFs | VictoryShares
WisdomTree wisdomtree.com | WisdomTree
Xtrackers etf.dws.com | Xtrackers

Sibling applications

Application Data provider Repository
AAM Official AAM catalog/detail HTML + full holdings XLS + SEC N-PORT holdings fallback + Yahoo market history/dividends AAM
abrdn (Aberdeen) Official Aberdeen gateway + SEC N-PORT holdings fallback + Yahoo history/dividends aberdeen
Amplify Amplify ETFs Firestore data feed + SEC EDGAR N-PORT-P holdings fallback + Yahoo Finance history/dividends Amplify
ARK Invest ark-funds.com fund pages + overview/NAV-history/performance JSON + official daily holdings CSV + SEC EDGAR N-PORT-P holdings fallback + Yahoo Finance distributions/history fallback ARK
Capital Group Official Capital Group fund data + SEC N-PORT holdings fallback + Yahoo history fallback Capital-Group
Fidelity SEC EDGAR N-PORT-P + Yahoo Finance Fidelity
First Trust ftportfolios.com official ETF list + fund summary, holdings, distribution and price-history export pages + SEC EDGAR N-PORT-P holdings fallback + Yahoo Finance history fallback First-Trust
Franklin Templeton franklintempleton.com ETF listings + product pages + SEC EDGAR N-PORT-P Franklin
Global X globalxetfs.com Next.js catalog and fund pages + dated full-holdings CSV Global-X
Goldman Sachs am.gs.com fund finder + detail pages + SEC EDGAR N-PORT-P Goldman-Sachs
Invesco invesco.com fund pages and sitemap + official Invesco fund API (monthly returns, NAV, AUM, yields, daily holdings, expense ratio) + SEC EDGAR N-PORT-P holdings fallback + Yahoo Finance history/dividends Invesco
iShares iShares (BlackRock) product workbooks iShares
JPMorgan am.jpmorgan.com fund explorer + product-data JSON JPMorgan
NEOS neosfunds.com lineup table + official fund pages + daily holdings CSV Neos
Northern Trust etfs.ntam.northerntrust.com funds list + per-fund CSV/JSON downloads Northern-Trust
Pacer ETFs paceretfs.com product catalog and fund pages (Cloudflare WAF; r.jina.ai proxy fallback) + SEC EDGAR N-PORT-P (Pacer Funds Trust) + Yahoo Finance history/dividends Pacer
Parametric eatonvance.com ETF catalog and Parametric product pages + SEC EDGAR N-PORT-P holdings + Yahoo Finance history/dividends Parametric
ProShares proshares.com ETF finder + fund pages + official data host ProShares
Schwab schwabassetmanagement.com product pages + CSV exports Schwab
SP Funds sp-funds.com homepage catalog, fund pages and daily holdings CSV + SEC EDGAR N-PORT-P holdings fallback + Yahoo Finance history/dividends SP-Funds
SPDR SSGA / State Street public feeds SPDR
Sprott ETFs sprottetfs.com fund pages + SEC EDGAR N-PORT-P (Sprott Funds Trust) + Yahoo Finance history/dividends Sprott
Tema ETFs Tema official fund pages + dated daily holdings CSV; SEC EDGAR N-PORT-P holdings fallback only + Yahoo Finance price/history/dividend fallback Tema
Themes ETFs themesetfs.com catalog + daily holdings CSV + Yahoo Finance history/dividends + SEC N-PORT-P holdings fallback Themes
VanEck vaneck.com ETF finder + product pages VanEck
Vanguard Vanguard product pages + SEC EDGAR N-PORT-P Vanguard
VictoryShares VCM VictoryShares catalog and product JSON + SEC EDGAR N-PORT-P holdings fallback + Yahoo Finance adjusted-market-price history VictoryShares
WisdomTree WisdomTree product table + SEC EDGAR N-PORT-P + Yahoo Finance WisdomTree
Xtrackers Official DWS catalog/US sitemap + PDP/XLSX + SEC N-PORT-P holdings fallback + Yahoo Finance daily prices/history/dividends Xtrackers

License

MIT - same as all sibling ETF repositories.

Global X® and the fund names/tickers referenced here are trademarks of Global X Investment Management LLC. This is an independent, unofficial tool; it is not affiliated with, endorsed by, or sponsored by Global X. All data is reproduced from Global X's own public pages and downloads, with public SEC EDGAR filings and Yahoo Finance as fallbacks, for research purposes. All other trademarks, including index names, are the property of their respective owners.

About

Global X ETF holdings-to-watchlist: static ./api/globalx feed generated by a Bun updater from globalxetfs.com Next.js catalog and fund pages, dated full-holdings CSV

Topics

Resources

Stars

0 stars

Watchers

0 watching

Forks

Contributors

Languages