alt Hacker NewsShow HN: Hacker News historic upvote and score data
Hi yall!
I've been using hacker news for a while but one of the things I started wanting recently was the ability to have alerts for any stories I post.
The thing that pushed me over the edge was Hackclub's shipwrecked https://shipwrecked.hackclub.com/ (hackathon in the boston bay for anyone that can make 4 projects over the summer and get at least one of them to go viral). One of the options for "going viral" was to get to the front page of hacker news but I was constantly scared that I would miss it getting on there lol so I whipped up a quick slackbot to send alerts to a channel. It was dead simple but it did work.
Once I had the bot I realized I could do wayyyy more with the data I was collecting so I decided to add some historical data initially thinking I would generate graphs and then embed them in the message but decided to quickly try using Bun.serve to host a quick dashboard mainly since I wanted to see how the developer experience was. Spoiler it is amazing. I've gotten really inspired by web components and the idea of only using universally supported `html`, `css`, and `js`. Bun results in an amazingly nice developer experience where you can just import the `index.html` and assign it to your root route and be done. Sorry for shilling about bun but it truly was one of my favorite parts of building this besides drizzle.
The dashboard has a graph of the points earned and position on the leaderboard over time (updated every 5 minutes) and then the expected stats like peak points, peak position, author, and comment count.
Also btw all the code is open source ofc on both my tangled repo (https://tangled.sh/@dunkirk.sh/hn-alerts) as well as a github repo (https://github.com/taciturnaxolotl/hn-alerts) and you can try the hosted version at https://hn.dunkirk.sh I'm planning to add the ability to just install the slackbot to any workspace and have workspace specific leaderboards but that will require a bit of refactoring and probably abandoning the slack-edge package.
Also you can view specific item's data by simply replacing news.yc.com with hn.dunkirk.sh like: https://hn.dunkirk.sh/item?id=44115853
Comments
Are you sure this data is correct ? There is a listing of the "Most Upvoted Story" at 390, but I feel like I have seen regularly items higher than that.
One example I see today is : https://news.ycombinator.com/item?id=44174965
with 393.
> Highest Points 461 Most upvoted story
Not sure that's accurate. If you look at just yesterday, you have this story: https://news.ycombinator.com/item?id=44163063
Which has over 2200 points.
Looking at the charts they all seem to take off. To me that begs the question, do they take off because they hit the front-page and get all the attention, or are some of them bought upvotes like on Reddit?
nice job! I once created [Track HN](https://track-hacker-news.com) which does similar things! I've already archived historical data of scores/comments/ranks. It seems our projects overlap - maybe we can combine our efforts to hack something even cooler :).
10 months is definitely not a while in my book.
Nice site, slick. Not sure what use the data has to me, but I don't make posts, so that's my fault.
It would be nice if the chart would move down or something when you select a post that's lower in the rankings. I had to scroll back to the top after clicking one of the bottom posts.
Nice site. Looks slick!
I wonder though how the data is actionable? What can I do with it?
According to your stats, when is the best time to post to get noticed?
This is done with claude . You know why I know. looks awesome.
The problem with analyzing HN comment and submission voting is we don't (AFAIK) have access to downvotes and that changes everything. We used to know the net votes for comments but that was removed years ago. Only submissions have net votes.
But what you find is that position matters, both for submissions and comments. There's a world of difference between being on the front page or not and being on the first page of cvomments (nad the higher on that page the better) for getting eyeballs (because people only scroll so far) and thus votes.
I mention this because both submissions and comments are heavily curated. It's more obvious with submissions where one post can stick around for 12 hours with 30 net votes on the front page while another will disappear within 15 minutes with 200+. Part of this is known. For example, the moderation team tends to dislike "political" submissions. I put that in quotes because what is and isn't "political" can be open to interpretation. Often on the Internet in general something is "political" if you don't agree with it and it's not if you are. You might even hear descriptions like "common sense" about the latter.
Comments are heavily curated too. Some of this is just rotating the top comments for freshness. Some of it is some sort of commenter rating where their comments will tend to be ranked high or low when fresh. And some of it is simply downranking certain comments.
On Reddit this would all be both more transparent and easier to visualize and analyze because we could sort by different critera, we could see total votes and we could see up/down vote breakdowns.
To be clear, no shade to the moderation team here. Just observing what happens.
One side note: I believe Shipwrecked isn't you but as a general observation to anyone, it reverses the mousewheel direction. Don't mess with scrolling.
Good work here. Just a few comments about your graph:
1. This may be personal preference but I would find the chart more reasonable if there was a horizonal line for where the cursor is. The way you have it, it feels a bit "disconnected" but maybe that's just me. I'm thinking of, say, how Google Finance does their charts;
2. You can probably compact the header into a single row/line;
3. I'm visually impaired and would like more contrast in the colours used accessibility-wise;
4. You have the submitter's name. That could be a link to their profilee;
5. This may just be preference but the tooltip feels "off" to me. This may be related to the above point about not having a line for the cursor maybe? But also the dark background doesn't seem to fit. Maybe it's the color too. I could just be nit-picking here;
6. Show number of comments maybe?
And the dashboard:
1. The cards themselves have a lot of wasted space. Again, you could link to the profile in them;
2. If you're scrolled down the page, when you click on a card you don't see the graph;
3. There's no hover over effects on the dashboard graphs. There easily could be I suspect;
4. I'm not sure a graph for raw score makes sense. It can only go up. Maybe show how many upvotes gained in that period?
5. The legend has different styling on the dashboard graph for some reason;
6. If you scroll down, click a card then scroll up you see the graph but you can't see what submission it belongs to. Maybe put some indicator on the graph or rethink how the layout works in general so you can see both the graph and selected card at the same time regardles of scrolling.
Anyway, good work.
I'm also collecting this data, and it allows rendering beautiful heatmaps like here: https://github.com/ClickHouse/ClickHouse/issues/59020
Try: ssh [email protected]
SELECT id, max(score) AS s, max(descendants), argMax(title, update_time), arrayStringConcat(arrayMap(x -> format('\x1b[38;2;{0};{1};{2}m\xE2\x96\x88\x1b[0m', least(greatest(sqrt(x / 10) * 255, 0), 255)::UInt8::String, least(greatest(sqrt((x - 10) / 50) * 255, 0), 255)::UInt8::String, least(greatest(sqrt((x - 50) / 250) * 255, 0), 255)::UInt8::String), arrayDifference(arrayFill(x -> x != 0, maxResample(now() - INTERVAL 2 DAY, now(), 3600)(score, update_time))))) AS bar FROM hackernews_changes_items WHERE id IN (SELECT arrayJoin(ids) FROM (SELECT ids FROM hackernews_top WHERE type = 'top' ORDER BY update_time DESC LIMIT 1)) GROUP BY id ORDER BY s DESC LIMIT 10
Malwarebytes warned me about this site.
Website blocked due to riskware. IP 37.27.51.34 port 443
Firefox error reported as Problem Loading Page:
>SSL received a record that exceeded the maximum permissible length.
Error code: SSL_ERROR_RX_RECORD_TOO_LONG
Could be my machine. I'm on Win7Pro with FF 115.24.0 so old and out of date here.