A lot of the sources I wanted had dropped their feeds and gone email-only, so I spent a while turning newsletters back into feed items. Sharing the parts that surprised me, since this sub has the same problem every week. Getting the mail is the easy half. An address per newsletter, pointed at a mailbox you control, and the address itself is the feed identity. No parsing needed to know which source a message belongs to. Then the problems start. Not everything that arrives is content. Sign up to one newsletter and you get a welcome mail, a confirm-your-subscription mail, a receipt if you paid, and a "here's your login code" weeks later. Dump all of it into a feed and the feed is garbage. I sort each message on the sender, subject and a snippet, and only the actual issues become items. Codes I pull out with plain regex and surface separately, because a verification code inside a "feed article" is useless. Double opt-in is the whole ballgame. Most newsletters send a confirmation link and nothing else until you click it, so if your address can't receive, or you can't get at the link, you're just not subscribed. Unsubscribing is better standardized than I expected. List-Unsubscribe has been around forever, and RFC 8058 adds List-Unsubscribe-Post for real one-click. Worth reading carefully though: you may only auto-POST when the header advertises one-click, otherwise you can trip a "confirm your unsubscribe" page and do nothing. HTML email is not article HTML. Tables inside tables, inline styles, occasional script tags. I strip the executable parts before storing and render in a WebView with JavaScript off and navigation locked, which is the actual control. Images I leave alone on purpose, since each sender has its own address and a pixel tells them only that this one address opened it. Replies need a target. If you want to reply from the address the newsletter wrote to, you need the sender address plus the Message-ID off the original, or the thread breaks in their client. Identity is the RSS problem all over again. Feeds lie about guids; email has Message-ID, which is better, but forwarding and relays mangle enough of it that you still want your own key per stored item. I build one of these (Rovatrove, Android) so take the framing with that in mind. Happy to go into any of it, especially if you've done the mail side yourself and found something I haven't hit yet.   submitted by   /u/rovatrove   to   r/rss [link]   [comments]