Email List Building and Analytics: Understanding Newsletter Engagement and Deliverability

A Confused Writers' Guide to Building Email Lists

TLDR: Newsletter analytics reveal critical insights about engagement patterns, email deliverability, and subscriber behavior. The evolution from keyword-based spam filters to engagement-driven algorithms means that list quality matters more than list size, and understanding your data is essential for growth.

Summary:

After noticing a decline in open rates, the author discovered that promotional emails were landing in the "promotions" folder, affecting deliverability. This led to building a Streamlit app for newsletter analytics that works for any Substack publication, revealing insights that the Substack dashboard doesn't show clearly.

The key finding: list engagement is about the same with 1,571 emails delivered at 40% open rates as it was with 192 deliveries at 55% open rates at the beginning. This demonstrates that subscriber count alone is misleading—engagement quality matters more than quantity.

The Substack dashboard has tracking issues. For example, it calculated a 30% open rate for the last post rather than the 40% found by looking at available email delivery data. The dashboard assumes emails were delivered to all 2,237 subscribers even though there's only delivery data for 1,571 emails. This creates a false picture of performance.

Email deliverability has evolved dramatically. Spam filtering has transformed from keyword-based blocking (1990s) to Bayesian filtering (2002) to engagement-driven intelligence (2019-present). Today, Gmail's algorithm prioritizes engagement signals over almost everything else. If a user replies to your email, it's the strongest signal that you belong in the Primary tab. If users delete your email without opening it, or report it as spam, your domain reputation tanks.

The analysis revealed that the author was only hitting three out of four main Substack growth levers consistently: Notes, App, and Your Weekly Stack. The notable absence is Substack recommendations, which was one of the main growth engines driving Lenny's newsletter to over $1M in ARR.

Surprisingly, the most engaged subscribers are those that subscribed via Your Weekly Stack—an algorithmically generated weekly list of personalized recommendations. However, all three Substack growth channels have mostly brought in free subscribers, which is great but limiting for monetization goals.

The engagement pattern is binary: you're either engaged or you're not. About 30% of the list is highly active, engaging with more than 80% of emails, but there's also a huge section (42%) that hardly engages with anything published. This suggests the need to remove inactive subscribers to improve deliverability and, hopefully, open rates and engagement.

Content strategy matters. Some posts had high "f**k-off" rates—percentages of subscribers for whom that post was the last email they ever opened. One post, "An argument against.." led to 15.7% of people who opened it never engaging again. This highlights the importance of understanding what content resonates versus what drives people away.

For architects and teams, this analysis demonstrates the importance of data-driven decision making. The default dashboards provided by platforms often don't tell the full story. Building custom analytics tools can reveal insights that drive better strategies. Understanding engagement patterns, deliverability factors, and content performance is essential for building sustainable growth.

Key takeaways:

• List quality matters more than list size for engagement
• Platform dashboards often have tracking issues that mask true performance
• Email deliverability is driven by engagement signals, not just content
• Binary engagement patterns suggest focusing on active subscribers
• Content strategy must balance attracting new readers with retaining existing ones

Tradeoffs:

• Removing inactive subscribers improves deliverability but reduces list size
• Focused content strategies improve engagement but may limit audience growth
• Custom analytics provide better insights but require development effort

Link: A Confused Writers' Guide to Building Email Lists

This article was generated from newsletter content. The summaries are based on extracted article content and may not reflect the full depth of the original articles.