Detecting relevant content of interest

When AI is actually a bit dumb. 

Consider something I'm seeing a lot:

• Content (e.g. news) app shows lots of different content.
• You read an article in the app within a specific category.
• Several hours later, an automated backend process tries to prompt re-engagement.
• The backend process looks at what categories of content you've looked at recently.
• It notices a recent article that is in that category and is getting lots of views.
• The backend process then sends a notification about that content as you're likely to be interested. (It knows you read content in the category, and lots of people are reading this new article. It should be of interest.)
• But the "assumption" was based on a lack of consideration for articles already read. (It was of interest, that's why you read it several hours ago.)
• Enough people click on these notifications to make them look like they're doing a good job of promoting re-engagement.
• People click on the notifications because they might be related to or following on from what they read earlier and not realizing that it is the exact same article.
• Analytics only tracks openings from the notifications, not how much time is spent reading the article they were using to promote re-engagement.

Analysis of analytics doesn't flag this, and the opacity of "the algorithm" doesn't make it clear this is what's happening.

All the while, many people get wise to these pointless notifications and turn them off, and so many miss something actually useful in the future.

I know there are more complex and varied scenarios than mentioned above, including how the above could be considered a successful "engagement" in other ways.

The main thing I take away from this is that I don't want to be creating software that tries to do something clever and useful without being able to accurately tell it is being successful and providing value.

Creating something that is trying to be clever but comes across as stupid because it doesn't make wise or appropriate use of the information it already has does not excite or interest me.