Saidit.net - Jan 14, 2020 - New updates, automatic title fetching, comment history lookback limit bug fixed

submitted by magnora7[A] from self.SaidIt

sorted by:
top
newinsightfulfun
you are viewing a single comment's thread.

view the rest of the comments →

[–]magnora7[S] 3 insightful - 2 fun3 insightful - 1 fun4 insightful - 2 fun -  (5 children)

No problem.

i think the unlimited history alone makes saidit more useful than the competition.

I agree. We intend to create something of historical record here, that will be as permanent as the internet allows. And full comment lookback is an important part of that historical record being accessible, so I couldn't be happier that d3rr got it working properly.

[–]useless_aether 4 insightful - 1 fun4 insightful - 0 fun5 insightful - 1 fun -  (4 children)

truly amazing. lucky, that text compresses extremely well! i just hope the search function works too, i almost never use it, but now i think i should.

[–]go1dfish 5 insightful - 1 fun5 insightful - 0 fun6 insightful - 1 fun -  (1 child)

I don’t think the Reddit/saidit codebase is set up for such compression.

I’ve switched notabug.io over to using gzip encryption on the live db in keeping with the aim of cheap operation. All of saidit’s content and nab’s content with indexes fits in 4gb of disk space this way while operating. Before these optimizations it was taking 15Gb on disk.

My indexing system in its current form is not well suited for historical access though, I use a limit of 1000 items for most listings to keep performance reasonable. This can likely be improved in the future with some sort of hierarchical date based indexing.

[–]JasonCarswell 2 insightful - 2 fun2 insightful - 1 fun3 insightful - 2 fun -  (0 children)

Date based indexing might also be good for incremental sharding, should you choose to share backups, decentralize, etc. rather than large complete-full-archive torrents.

(I'm talking abstractly out my ass in general concepts and have no idea how to practically do this in code.)

[–]magnora7[S] 3 insightful - 1 fun3 insightful - 0 fun4 insightful - 1 fun -  (1 child)

It works pretty well, but it's not perfect. The search has lots of options you can view beneath the search bar, to help you refine your search.

And if all else fails, you can just use google or duckduckgo with the qualifier "site:saidit.net" as part of your search.

[–]useless_aether 3 insightful - 1 fun3 insightful - 0 fun4 insightful - 1 fun -  (0 children)

site:saidit.net

ah yes, should def use that. thanks again.