all 41 comments

[–]shitfourbrains[S] 4 insightful - 2 fun4 insightful - 1 fun5 insightful - 2 fun -  (17 children)

all these bitches at reddit are going on and on about how they cant use an api because it is so expensive.

fk an api, i say..

if the browser can read and render it, then a programmer can scrape it and you can automate the dom.

[–]ID10T 1 insightful - 1 fun1 insightful - 0 fun2 insightful - 1 fun -  (16 children)

It's not feasible. Scraping breaks when there are design changes. Any change to the page templates can break scraping so you would constantly being playing catch up to fix issues all the time.

Then there's the other problem of security. In order for a scraper to access your account, messages, subscribed subs, etc., it would have to login as you. With an API you can use an identity platform like OAuth to provide authentication with sharing your credentials with the app itself. With a scraper you would have to give it your login to use as you.

Which means that the owner of the scraper would have access to all the user logins and could potentially take over all those accounts, locking the users out of their own accounts.

Sounds like a nice honeypot to create a botfarm with many established user accounts now that you mention it. Imma get right on this. I know a guy in Belarus that will pay top dollar for Reddit accounts 😂

[–]transbob 1 insightful - 1 fun1 insightful - 0 fun2 insightful - 1 fun -  (15 children)

Scraping breaks when there are design changes.

That is a statement that is true and not true at the same time.

If you were an experienced programmer you would know that there are ways to get around such things.

My JavaScript skills are somewhat limited at this time, but I've been programming for 40 years and I've been a c programmer for 30 years.

It is the programmer's job to know how to get around obstacles.

[–]ID10T 1 insightful - 1 fun1 insightful - 0 fun2 insightful - 1 fun -  (14 children)

My experience is not important to what I'm saying, but it compares with yours with the difference being that I have built and used scrapers in a production environment. This is what happens when you rely on scraping a site out of your control.

[–]transbob 1 insightful - 1 fun1 insightful - 0 fun2 insightful - 1 fun -  (13 children)

II have built and used scrapers in a production environment.

My first web scraper that I wrote was written entirely in C language using sockets code where I scanned through all of the different cities of Craigslist for certain keywords and then I generated a list of potential job listings that I might be interested in.

My first actual scraper was not a web scraper but instead it was a war games dialer where I would dial sequentially through phone numbers looking for computers that would answer with a carrier signal and then it would save that phone number to a five and a quarter inch floppy disk text file.

I've been riding code since 1983.

I could write a bot on Reddit or Twitter whether they like it or not and I don't need no stupid API.

[–]ID10T 1 insightful - 1 fun1 insightful - 0 fun2 insightful - 1 fun -  (9 children)

Lol okay. Like I said, my experience is similar to yours. Argument from authority is a logical fallacy. And you suffer from knowledge transference. The only way to stay current in programming or any other technical profession that rapidly advances is to be humble and continue learning.

If you dispute a statement I made you can present a valid argument against it other than "I've been programming since the 80s".

Does your Craigslist scraper still work? DOM changes on elements that you are scraping would break it.

[–]transbob 1 insightful - 1 fun1 insightful - 0 fun2 insightful - 1 fun -  (8 children)

Being humble don't mean nothing as a programmer.

Maybe you think that bank robbers need to be humble, lol.

I don't need to present arguments in a way that you find to be suitable.

Does my Craigslist crawler still work? No but I don't expect it still work it was an app that I wrote and I used it and then I was done.

I don't work for those faggots at Facebook.

Do you know why people argue with me all day long and so often they tell me that experience doesn't mean anything and it's usually kids with a couple of years experience that tell me that?

[–]ID10T 1 insightful - 1 fun1 insightful - 0 fun2 insightful - 1 fun -  (7 children)

Ok tough guy 😂 but I have more experience than you. And the fact your scraper doesn’t still work proves my point.

I also am happy to listen to programmers with just a few years experience.

Being humble is what allows you to keep learning instead of thinking you know better than everybody.

Sometimes they have really good ideas I didn’t think of. But you do you.

[–]transbob 1 insightful - 1 fun1 insightful - 0 fun2 insightful - 1 fun -  (6 children)

Sometimes they have really good ideas I didn’t think of.

Go f*** yourself

[–]ID10T 1 insightful - 1 fun1 insightful - 0 fun2 insightful - 1 fun -  (5 children)

Haha real mature fifty year old behavior lol

[–]Zapped 1 insightful - 1 fun1 insightful - 0 fun2 insightful - 1 fun -  (2 children)

I've been riding code since 1983.

Go get 'em, cowboy.

[–]transbob 1 insightful - 1 fun1 insightful - 0 fun2 insightful - 1 fun -  (1 child)

I use my microphone to type and it's always hearing the word "riding" when I really said "writing"

Oh well

[–]Zapped 1 insightful - 1 fun1 insightful - 0 fun2 insightful - 1 fun -  (0 children)

Yes. I enjoy these mis-readings.

[–][deleted] 2 insightful - 2 fun2 insightful - 1 fun3 insightful - 2 fun -  (1 child)

It needs some semi colons and shit

[–]transbob 2 insightful - 1 fun2 insightful - 0 fun3 insightful - 1 fun -  (0 children)

Good point.

[–]SMCABwhatever 1 insightful - 3 fun1 insightful - 2 fun2 insightful - 3 fun -  (4 children)

Sooooooo........ transbob is shitfourbrains?

I just don't get it? Why?

[–]transbob 1 insightful - 2 fun1 insightful - 1 fun2 insightful - 2 fun -  (3 children)

If you knew my other 15 usernames... With those usernames, I'm able to form a brigade and argue with you and drive you nuts.

The key is to never let you know what those other usernames are.


[–]SMCABwhatever 1 insightful - 1 fun1 insightful - 0 fun2 insightful - 1 fun -  (2 children)

Well I don't argue unless I'm bored, and that's pretty rare, so, do whatever you gotta do pal. Seems worthless to me. Just another fatfuck withering away on the internet. Don't forget to order that body pillow with the Japanese girl on it, loser.

Plus, you pathetic multiple username fags always slip up. You can't help it. Your mental illness always gets in the way.

[–]transbob 1 insightful - 1 fun1 insightful - 0 fun2 insightful - 1 fun -  (1 child)

Yeah that's all fine and good.

Feel free to block me and live your life in pleasure.

Feel free to block all 27 of my usernames.

[–]SMCABwhatever 1 insightful - 1 fun1 insightful - 0 fun2 insightful - 1 fun -  (0 children)

Sorry, you've reached customer service. Your tracking number for your new waifu body pillow is 1P86G83KG98379L927

[–]LarrySwinger2 1 insightful - 2 fun1 insightful - 1 fun2 insightful - 2 fun -  (1 child)

Are you not even including me in this list? I'm disappointed, Ed.

[–]transbob 1 insightful - 1 fun1 insightful - 0 fun2 insightful - 1 fun -  (0 children)

Obviously you were not online at the time when I ran the script.

I think I ran it shortly after the chat room does it's reset twice a day or is it four times a day I can't remember it does it at 1:00 and 7:00 so I think it does it four times a day but I can't say that for sure about that.

[–]Canbot 1 insightful - 1 fun1 insightful - 0 fun2 insightful - 1 fun -  (1 child)

A bot you say?

I like it!

Why did you pick java? What features would you like to implement?

[–]transbob 1 insightful - 1 fun1 insightful - 0 fun2 insightful - 1 fun -  (0 children)

There is a difference between Java and JavaScript, though they are similar languages they are not the same.

What features would I like to implement? It doesn't really matter I'm just polishing my skill set and learning how to read and write the DOM.

[–]Vulptex 1 insightful - 1 fun1 insightful - 0 fun2 insightful - 1 fun -  (6 children)

What does this data mean?

[–]transbob 1 insightful - 2 fun1 insightful - 1 fun2 insightful - 2 fun -  (5 children)

These are the current users that are logged into the chat, at least they were at the time when I ran this script.

This gives me the ability to monitor the chat users and to make a graph that shows how many hours per day each user stays online how many posts they make per hour etc.

Many users do not post in subs that are readable in New or whatever and so you don't really see their usernames by cruising through the publicly available posts or comments. If I snatch up these usernames from this list then I can look at their posting history and I can find all of the hidden subs on this site.

Not only that but I'm also making others aware of this potential program that can be written.

I am opening a can of worms that you cannot close

[–]Site_rly_sux 2 insightful - 1 fun2 insightful - 0 fun3 insightful - 1 fun -  (3 children)

I have never logged into any chat on saidit ever. The idea of engaging any of these bigbrains in a casual chat makes me feel ill

[–]transbob 1 insightful - 2 fun1 insightful - 1 fun2 insightful - 2 fun -  (2 children)

I have never logged into any chat on saidit ever.

You did however visit the homepage of this website.

Anybody that does visit the homepage of this website I can grab your username from the chat.

And even if they stop posting those list of online users I can still get your usernames,

In 1983, The very first program that I ever wrote used Atari basic with line numbers and a 300 baud modem that connected through my joystick Port of my Atari 800. I designed and wrote a war games dialer, for which I was paid a visit by two federal agents at my apartment. They asked me politely if I would find a new hobby and I agreed.

[–]Site_rly_sux 2 insightful - 1 fun2 insightful - 0 fun3 insightful - 1 fun -  (1 child)

Wow and you went from doing that, to now scraping a list of 60 concurrent users of a niche q-adjascent hate speech reddit clone.

What an amazing life you must have lived homie

[–]transbob 1 insightful - 1 fun1 insightful - 0 fun2 insightful - 1 fun -  (0 children)

Whenever I study programming, I do it by writing a program.

So when I want to become familiar with JavaScript and how to read and write the DOM, I write some program that does something.

I'm so happy you approve

[–]Vulptex 1 insightful - 1 fun1 insightful - 0 fun2 insightful - 1 fun -  (0 children)

How exactly is this a can of worms to track how active saidit users are?

Also, the names in chat don't disappear until chat resets, even if that user logs off. So you can't really track how much time people spend here.

[–]Vulptex 1 insightful - 1 fun1 insightful - 0 fun2 insightful - 1 fun -  (2 children)

JavaScript doesn't sound like a good choice for bots to me. It's designed for manipulating HTML. It's also hellish in general.

[–]transbob 1 insightful - 1 fun1 insightful - 0 fun2 insightful - 1 fun -  (1 child)

Oh you are absolutely wrong about that. If you've never f***** around with devtools, then you don't know that JavaScript is the language that it was designed to be programmed in.

JavaScript is absolutely the language of choice to use when designing web automation tools.

[–]Vulptex 1 insightful - 1 fun1 insightful - 0 fun2 insightful - 1 fun -  (0 children)

Only if what you're looking for is Brainfuck Lite.

[–]transbob 1 insightful - 1 fun1 insightful - 0 fun2 insightful - 1 fun -  (0 children)

Probably one of the first apps that I will write will be to monitor the chat of various subs and to compile a complete list of as many usernames as I can get my hands on.

From that point I will read through all of their posts and comments and I will compile a list of unknown subs that don't appear on the homepage or on the new page.

And why do I do this it's just how I get my feet wet with JavaScript.