you are viewing a single comment's thread.

view the rest of the comments →

[–]magnora7[S] 4 insightful - 1 fun4 insightful - 0 fun5 insightful - 1 fun -  (6 children)

Thanks. I always figured, what's the point of all this open-source code if no one ever uses it?

Multiple sites with different approaches to tech and policy is itself a form of decentralization even when not mediated by software.

I couldn't agree more. That's why I like notabug and other "competitors" to saidit. Together we form a network that itself is distributed. You're more than welcome to replicate any rules we have here, I think our system has worked out pretty well so far.

Running the reddit instance is tricky at times, but once you get the right OS and configure it correctly, it's doable. It seems to run fine on a small machine. We did have to upgrade from 4GB to 8GB RAM because we were running out of memory. Mostly due to the cassandra memory management system that keeps everything in RAM. Have you worked with this?

I'm glad you like the no-downvote system too. It also allows filtering by fun vs insightful vote types, which will be useful when cat pictures inevitably take over the website. Haha.

[–]go1dfish 4 insightful - 1 fun4 insightful - 0 fun5 insightful - 1 fun -  (5 children)

I've not messed with cassandra no.

Notabug is built on gundb, but uses it mainly as a communications protocol. LMDB is the backing store (used to be redis) and it's worked quite well for my purposes so far.

Each VM only has 2gm of ram right now, one deals with web socket connections and server side rendering and the other indexes content as it comes in (the indexer has password/secret key for so like to keep it isolated, frontend web server has no secrets at all)

The indexer also copies the database data to a openstack-swift server (think open source amazon s3) and most all of the data queries go direct to that and never hit my server, so web socket server is really only used for updates and not all queries (as is typical with gun and how I originally started)

[–]magnora7[S] 3 insightful - 1 fun3 insightful - 0 fun4 insightful - 1 fun -  (4 children)

Oh wow so you've changed quite a bit about the backend then.

Our thinking was that even if it's complicated to run on a small server, it's already had 7 years of testing and fixes for a large-scale environment, so if it ever took off we'd be equipped to scale it easily. I think we ended up making things easy on ourselves thankfully.

2GB of RAM is impressive, I think our deployment takes up 3GB minimum. Your 2-server setup is interesting, the design sounds useful so one server can be a dedicated "gatekeeper". Do you use cloudflare or any DDOS protection, or do you do that yourself too?

Our cloudflare servers do a similar thing to your data query system, I think cloudflare actually ends up directly serving about 60% of our data or so, just mirroring what we already have. So it's probably not as robust as yours, but ours also only serves updates in a way. And cloudflare acts as a globally distributed server network. It's pretty good for being free. But it's not open-source, and we do sacrifice a bit of control over our system for the DDOS protection and distributed cached serving. But it seems to have been worth it so far. d3rr just wrote a script to scrape the original (pre-cloudflare) IP addresses from the header information and add those to our DDOS blocking list and ban individual IP addresses. So we've been able to work around the shortcomings.

[–]go1dfish 3 insightful - 1 fun3 insightful - 0 fun4 insightful - 1 fun -  (3 children)

Oh wow so you've changed quite a bit about the backend then.

There is no python code at all, backend is totally new from scratch with only the HTML/CSS from reddit used.

My host provides DDOS protection but I still run cloudflare in front mostly for convenience.

Cloudflare's cacheing is active somewhat on the data requests I mentioned earlier as well. doesn't use server side rendering, even if both my vm's go down it will still be readable (it's hosted entirely on openstack-swift)

Cloudflare is definitely a tradeoff and something the users of notabug are often critical of. It's been worth it in my case as well but I can certainly understand the criticisms of it.

[–]magnora7[S] 2 insightful - 1 fun2 insightful - 0 fun3 insightful - 1 fun -  (2 children)

Wow impressive about the backend. How many man-hours do you think it took you to get the new backend working?

[–]go1dfish 3 insightful - 1 fun3 insightful - 0 fun4 insightful - 1 fun -  (1 child)

First version I got running in a week maybe 20 hours, but have been doing 10 hours or so a week on it in the year since.

Still not very feature complete, but making progress

[–]magnora7[S] 2 insightful - 1 fun2 insightful - 0 fun3 insightful - 1 fun -  (0 children)