Methodology
Which sites are read
TodayNS keeps a candidate list of news sites, ordered by approximate traffic. Each week it checks every candidate and reads the first 50 that pass all of these:
- its robots.txt lets the TodayNS bot read the page used,
- it has not opted out of text and data mining (a TDMRep file or response header),
- its RSS feed, news sitemap or section page returns headlines.
These rules are checked again on every collection, so a site that adds a restriction is dropped immediately. Some well-known sites are listed with a link only, marked “Not scored”, while TodayNS asks for their permission.
What is collected
Only each headline, its link and its time. Articles are never opened.
How a headline is scored
An AI model (Claude Haiku 4.5, on Amazon Bedrock) gives each headline an integer from 0 to 100, with 50 neutral. It isn’t told which site the headline came from, so the same headline gets the same score wherever it appears. Each headline is scored once, and the score is saved.
How the scores are combined
- A site’s score: the average of its headlines seen in the past 24 hours, front page and business section separately.
- Overall news tone: the average of the sites’ front-page scores. Every site counts equally. A site needs at least 5 scored headlines to count.
- Financial news tone: the same, from business sections.
- Aggregators: sites that republish others are listed but left out of both, so a story is not counted twice.
When scores are taken
A new reading every 15 minutes, plus a fixed morning report at 5:00 AM Eastern. Each reading is saved permanently, with the time it covers and the time it was published, and is never changed afterwards.
How top stories are found
After every reading, each headline from the past 24 hours is turned into a numerical summary of its meaning (an embedding, from Amazon Titan Text Embeddings V2, which works across languages). Headlines whose meaning is close are grouped together. A group becomes a story when at least three different outlets carry it, and stories are ranked by how many outlets carry them.
The title and summary of each story are written by an AI model (Claude, from Anthropic) in its own words, from the grouped headlines. The model also removes headlines that are not about the story. TodayNS never publishes the headlines themselves; each outlet’s row links to its own article.
Own reporting or wire copy. An outlet is marked as carrying wire copy when its headline uses the same wording as another organisation’s headline for the same story. This is a simple test: it can miss rewritten wire stories, and it does not know which outlet wrote the words first.
Effect on scores. The overall and financial tones are recalculated from the same saved reading with the story’s headlines left out, and the two results are compared. Nothing is scored again. A difference smaller than half a point is shown as “no measurable effect”. A story with many headlines can still have little effect if its headlines are close to the outlets’ usual tone.
How the tone is compared with the market
On trading days, the section “Today vs. the market” compares the change in each tone since the 9:30 AM ET open with the change in the S&P 500, the Nasdaq-100 and the Dow over the same time. The index moves are measured with SPY, QQQ and DIA, the exchange-traded funds that track those indexes, using quotes from Finnhub. At 4:00 PM ET the comparison is fixed as the day’s closing scorecard. The session average is the average of the readings taken between the open and the close.
A single day shows what happened, not a pattern, and moving together does not mean one caused the other. Nothing here is investment advice.
Limits
Headlines are not articles, and tone is not accuracy. Automated scoring makes mistakes, especially with sarcasm, wordplay and some languages.