Contents
There is no time to read the timeline
You open the timeline on the commute. By lunch a different topic has taken over, and by evening you cannot say what the discussion was actually about. Following more people adds information without adding reading time.
The problem splits into three stages.
- Discovery: you cannot tell whether a post matters until you open it.
- Reading: even when it matters, long threads and quote chains take time.
- Digesting: you still have to condense what you read into “here is what happened today.”
TopicDigestX moves those three stages to a server. You register keywords as topics, collection and summarization run automatically, and you read only a digest. What exists today is the foundation; collection and summarization are still ahead.
Why existing tools fall short
| Approach | What it gives you | Where it breaks |
|---|---|---|
| Reading the timeline | Full context | The volume does not shrink, and the order disappears as it flows |
| Lists and search | Catches accounts you do not follow | The reading is still yours to do |
| Aggregators and newsletters | The highlights, already summarized | Not tuned to your topics, and you cannot pick the granularity |
| Bookmarks and read-later | You can find it again | It piles up; the reading still happens later |
| Pasting posts into an LLM | A summary on demand | Manual copying every time, and that one batch only |
None of these removes the reading itself. What can be removed is the searching before, and the condensing after.
The design
Here is the workflow before and after.
Figure 1: Knowing what is trending, before and after
Three decisions define the design.
- Store the interest: keywords become a row in the database, not something retyped each session.
- Move the expensive work server-side: collection and summarization are detached from the reader’s waiting time.
- Fix the output shape: you decide how much to read before you start reading.
The trade-offs are real. Freshness depends on how often collection runs, so breaking news will lag. Summarization compresses, so nuance and counterarguments get flattened. What you buy is the ability to judge whether something is worth reading before spending the time.
Implementation notes
1. Turn an interest into one row
# backend/model.py
class Topic(Base):
__tablename__ = "topics"
id = Column(Integer, primary_key=True, index=True)
name = Column(String, index=True, nullable=False)
keywords = Column(Text, nullable=True)
Keywords are a comma-separated Text column. Normalizing search conditions into their own table can wait until topics actually carry more than one shape of condition.
2. Keep keys and connection strings on the server
# backend/main.py
class Settings(BaseSettings):
database_url: str = "sqlite:///./test.db"
llm_model_name: str = "gpt-3.5-turbo"
model_config = SettingsConfigDict(env_file='../.env', extra='ignore')
The browser never receives a key, and /config returns only a type name and the model name. When collection and summarization live on the server, that line is the security boundary.
3. Decide the output shape first
/digest/{topic_id} returns dummy data today, but the shape is already fixed.
# backend/main.py
return {
"topic_id": topic_id,
"generated_at": "2023-10-27T10:00:00Z",
"summary": "LLMs continue to advance rapidly. ...",
"key_tweets": [{"id": "12345", "text": "...", "author": "userA"}],
}
Fixing the shape lets the UI and the summarizer be built separately. Dummy content still supports a real screen as long as the interface is stable.
Results
The project is in development, so there are no user counts or time-saved figures. What can be claimed is qualitative.
- Discovery becomes “register a keyword” instead of scanning.
- The amount read is set by the digest length, not by how busy the timeline is.
- Condensing moves to the server in the form of a summary.
| Stage | Before | After (as designed) |
|---|---|---|
| Discovery | Scanning the timeline by eye | Collecting against stored conditions |
| Reading | Following posts and threads | Reading a summary and the key posts |
| Digesting | Condensing mentally | Saved as a digest and revisited |
The “After” column is the design target. Until collection and summarization exist, it stays unfilled.
When not to use it
- Breaking news: collection interval decides the lag. Real-time needs a different tool.
- Cases where the exact wording matters: summarization drops context around quotes and rebuttals. The design should let you jump back to the source through
key_tweets. - Topics with little activity: a niche keyword produces an empty digest, so a minimum-volume threshold is needed.
- Anything depending on the X API: changes to the API or its limits stop collection. The client needs to be replaceable.
- Multi-user use: there is no authentication yet, so topics are not owned by anyone.
- Schema changes: without Alembic there is no migration history.
Takeaways
- Do not try to add reading time; move the stages before and after reading to a server.
- Once an interest is data, collection and summarization become automatable.
- Fixing the output shape lets the UI and the processing be built in parallel.
Today the container is there and the digest is empty. The shape — decide what to read, then read it — does not change once collection and summarization arrive.
Related posts
- Building the TopicDigestX Backend on FastAPI and PostgreSQL
- Summarizing an X timeline automatically: designing a topic digest system with FastAPI and an LLM (in Japanese)
Building an information pipeline or an LLM summarizer of your own?
I take on technical consulting and development support.
See also
- The Small LLM That Answers Before It Thinks: Four Roles and a Comparison Design
- Co-evolving Four Roles, GAN-Style: The Testbed Behind umigame_GAN
- When the Timeline Fills Up with Reposts and Copies: Designing a Filter That Only Changes What You See
- Folding the Search for a Repost's Origin into One Query
- Finding the Original of a Reposted Image: How Origin Finder Combines Five Reverse Image Searches