all 20 comments

[–]d3rr 5 insightful - 1 fun5 insightful - 0 fun6 insightful - 1 fun -  (16 children)

Yeehaw it's an honor to be there man. I dig your mini reddit site you have going on. The spider button that has the article text is nice.

[–]TheWebOfSlime[S] 4 insightful - 1 fun4 insightful - 0 fun5 insightful - 1 fun -  (15 children)

Yeah, hopefully the formatting will improve, but it is mainly to be able to find connections in an automated manner and map it out. I hope to be able to have a third button that just plops a screen grab of the article, that works the same way as the drop text box, like this:

https://i.imgur.com/0a8WxSi.png https://i.imgur.com/SCcDbTQ.png https://i.imgur.com/xLU9Dz6.jpg https://i.imgur.com/KP2K2TO.jpg https://i.imgur.com/jDVQkSv.jpg https://i.imgur.com/1Wm15OJ.png https://i.imgur.com/g0Ixeto.jpg https://i.imgur.com/9hXRNKn.png https://i.imgur.com/pvhrVYI.png https://i.imgur.com/uCpu5du.jpg https://i.imgur.com/s2NX4M5.png

Everything is getting archived in two spots, in the background, two, upon creation of the link.

I already have it so I could, in one click, post to nearly every platform; but it is easy to drown out the smaller ones if I give any one place too many links in too short of a time.

[–]JasonCarswell 2 insightful - 1 fun2 insightful - 0 fun3 insightful - 1 fun -  (14 children)

Wouldn't a web scrape be more efficient than a large image, perhaps displayed in a frame (and also text copy/paste/searchable)?

I love the archiving too.

What do you mean "drown out the smaller ones"?

Also, do you have a RSS feed? (For others to read/download/collect/backup your content.)

[–]TheWebOfSlime[S] 3 insightful - 1 fun3 insightful - 0 fun4 insightful - 1 fun -  (13 children)

webofslime.com/rss

Text is more efficient, but getting it all to line up and look as nice as the author's intended is difficult without human participation.

Saidit is one of the smaller Internet communities. If I turned a script to saidit, my posts would be all anyone saw. On Voat, it only takes about 100 posts per day to be the number one poster. I have accounts on reddit that are doing thousands per day.

It wouldn't be fair to the members of the community or the owners to expose them to that type of firehose volume.

[–]wizzwizz4 3 insightful - 1 fun3 insightful - 0 fun4 insightful - 1 fun -  (6 children)

SVG is a better image format, because the text is readable.

[–]TheWebOfSlime[S] 2 insightful - 1 fun2 insightful - 0 fun3 insightful - 1 fun -  (0 children)

I will definitely look into that. Ty.

[–]JasonCarswell 1 insightful - 1 fun1 insightful - 0 fun2 insightful - 1 fun -  (4 children)

For this? No way. More complicated more like.

And it's not searchable if it's outlines. (I'm pretty sure you mean searchable not readable, because the screengrab image is readable.)

[–]wizzwizz4 2 insightful - 1 fun2 insightful - 0 fun3 insightful - 1 fun -  (3 children)

because the screengrab image is readable

By you. I take it that you're not a web designer. Well, these things have to be taken into consideration; not everyone reads websites using their eyeballs, or in exactly the same way as the CSS says to render it.

[–]JasonCarswell 1 insightful - 1 fun1 insightful - 0 fun2 insightful - 1 fun -  (2 children)

I don't know what your point is.

I have done web design but not too complex. I used CSS a dozen years ago and a variant here on SaidIt. I know what CSS is. I know what SVG is. I don't see how they mesh.

I understand the small-sized text-based web-scrape may not render its CSS perfectly and a large-sized screengrab would capture the render properly, but I don't see how a SVG scalable vector graphic could be generated to properly capture the CSS cascading style sheet information any better - all just to reduce the size rather than a screen grab. I would assume the images, whether JPG, PNG, GIF, and/or SVG would be downloaded separately and embedded in the code, just like the HTML web-scrape, but also assumedly into your SVG layout?

If this is true I'd love to learn about it and update my limited guestimated opinion on SVG web-scrapes including accurate CSS translations / rendering.

[–]wizzwizz4 2 insightful - 1 fun2 insightful - 0 fun3 insightful - 1 fun -  (1 child)

See how web-capture.net does it. It uses something something Qt something to do it; that's not perfect, as you can see by plugging this page into it (it doesn't support more recent CSS like animations) but the text is readable by a screenreader in certain circumstances.

[–]JasonCarswell 2 insightful - 2 fun2 insightful - 1 fun3 insightful - 2 fun -  (0 children)

I used web-capture.net to grab https://saidit.net/s/DecentralizeAllThings/ just because I had it open there. This isn't an image heavy page, but it has several small images, a banner, and footer.

  • .BMP = 11,554kb
  • .BMP.ZIP = 11,555kb
  • .JPEG = 1,058kb
  • .JPEG.ZIP = 1,058kb
  • .PDF = 394kb
  • .PDF.ZIP = 394kb
  • .PNG = 1,015kb
  • .PNG.ZIP = 1,015kb
  • .PS = 394kb
  • .PS.ZIP = 394kb
  • .SVG = 6,092kb
  • .SVG.ZIP = 6,092kb
  • .TIFF = 15,406kb
  • .TIFF.ZIP = 15,406kb

LOTS OF SURPRISES.

The SVG was much better than I expected, however there were several serious problems. Backgrounds were a problem, going beyond the frame was another, there were different defaults that you might expect would happen viewing a scraped HTML page. The gradients failed right off with a warning opening in Adobe Illustrator, though I don't know where there were gradients in the page (the ones that are there are pixel based images). While I'd hoped the SVG would have fields of text to be smaller, they went for an outline every single letter shape rather than just having the letter - thus why this SVG file is so ridiculously huge.

I was surprised the TIFF was bigger than the BMP. I haven't used TIFF since 2000 or so when PNG was adopted by animation studios and replace SoftImage's .PIC with better lossless compression. I knew they'd both be big.

I was also surprised the PNG was smaller than the JPG but there wasn't much imagery here.

I am hugely impressed that the PDF and PS were tied for way smallest, thought I haven't looked either. I've never even useds PS post script and I generally avoid PDFs.

Maybe most surprising was that the Zip didn't compress anything, though maybe it wasn't set to, not even the BMP or TIFF.

And for fun I CTRL-S saved from my Brave Browser the "Webpage, complete .html" The HTML file is 221kb and the folder is 765kb and together using 7-Zip with the default settings compressed down to 227kb.

So today I learned that folks can do more with SVG if they really want to, no matter how silly. Don't get me wrong, I love SVGs but I wonder if they've been "discovered" too late in the game. 20 years ago it might have had a chance to compete or merge or work in combinations with the more common web page things like GIFs, JPGs, HTML, CSS, and Flash (all rendered locally with faaar less powerful computers) but now the standards and browsers are even more entrenched, and failing revolutionary paradigm shifts it's moot. Things can change, like Firefox and all browsers are more like Chrome than ever, sadly - but at least ve have ze conformity!

I love SVG, but sometimes more is not better. d3rr and I were looking at emoticons, and the latest 4th generation batch had gradients. I preferred the 3rd gen designs way more, without gradients. Because they were poorly utilized. Maybe 5th gen will be better. I've see lots of supreme use of gradients since they were invented.

Anyway, neat stuff.

Socialism! Victory! Glory!

[–]JasonCarswell 2 insightful - 1 fun2 insightful - 0 fun3 insightful - 1 fun -  (5 children)

Fair point on the aesthetics. If size is no issue then a text scrape might be useful too, if for no other reason that searchability.

Do they count who is number 1? Does SaidIt count? Do people care?

Why are you funneling the internet through Reddit? What's your motivation? Have you been doing this long? I kinda get it but I kinda don't.

I kinda get it because I have a similar impulse, though not for so many. I was even considering starting a "SaidItBots" sub to start discussing how to DIY bots on SaidIt. While this could be opening a whole can of worms, I think that with adequate rules, openness and transparency we can deal with issues as they arise. One rule might be that all bots should have a dedicated sub feed with "Bot" in the title to indicate that it's bot posted. I have lots of YouTube channels I would do this for, thought many of them only post once or twice a month, while some post a few a week, and a few are daily. For example, "TheJimmyDoreShow_Bot" might the name of an active a subsaidit, about one every day or two on average, sometimes skipping a few days, sometimes 3 in one day.

Another thing the "SaidItBots" sub could do is vote on what bots are allowed. Maybe most folks don't want Jimmy Dore all the time. Or maybe they would do a probationary trial run and vote on whether it stays or goes.

Also, while I use only this user alias, it might be nice to set these bots up under different conditions, whether that means I'd create a "TheJimmyDoreShow_Bot" alter alias to post under, and maybe /u/magnora7 and /u/d3rr could re-route messages from /u/TheJimmyDoreShow_Bot to /u/JasonCarswell where I can answer them as myself not the auto-bot. Or maybe there's a way to share bots like multiple sub moderators.

I like the idea of automating because I'd rather spend more time interacting with folks than posting what a bot could do. Often, by the time I've finished posting the video is done and I'm doing another.

By making a rule that the bots need a dedicated sub with "bot" in the name, means it limits the number of bots for less firehose volume. This is both good and bad. We only have one sub a week and if ReallyGraceful only produces one or two videos a month, then that's much less incentive for me to bother with. Now there are several amazing channels out there producing at the same rate, and if I got 10 of them into one "VideoEssays_Bots" subsaidit, then it might be worth it and you'd get a new video once or twice every few days.

Are there any good tutorials or sources for setting up bots?

Also, is there a way to skip the SaidIt "new" list? For example, I just posted a Ben Swann video on YouTube and I didn't bother to isolate it from the playlist it was in (linked to from his site) and it would be nice for search and archival purposes to add the other 54 videos in that list - with a bot - but not if it's going to firehose folks. This would useful on countless other channels. In a similar vein, I use "YouTube-DLG" an open source downloader with a GUI that can download a single video, a playlist, or an entire channel (with thumbails) from most media platforms - and auto-renames them to your custom specs - super simple (but takes up far more space than just links).

[–]TheWebOfSlime[S] 2 insightful - 1 fun2 insightful - 0 fun3 insightful - 1 fun -  (4 children)

I run a lot of what I do through reddit for a number of reasons, but mainly because nobody notices if I have 1000 scripts all doing different things.

I need a lot of information to measure real human interest and artificial interest. This way, the only things that reach the top pade of my customized feed is actually good. There is a threshold between something that is interesting and something that is getting immediately upvoted by 1,000 shills in the first 5 minutes. I like to focus on the sweet spot.

In doing this, I circumvent the hypertargeting by mainstream media and Internet shills. I create my own upstream.

For your purposes, it sounds like you would want to use IFTTT and connect it to youtube and Google sheets. You can make an applet where every time you "Like" a video, it automatically posts it to a spreadsheet. You can connect to Reddit to post it to reddit, too, or make a second applet that "every time posts row to sheet, post to reddit." In Google sheets, you can add a row that concatenates the link and title into a saidit link. Then, you can pick the best and manually Ctl C Ctl V, or you can script within your sheet to post automatically to saidit.

[–]JasonCarswell 2 insightful - 1 fun2 insightful - 0 fun3 insightful - 1 fun -  (3 children)

TIL 2 maths.

I'd never heard of "pade" before. https://en.wikipedia.org/wiki/Pad%C3%A9_approximant

Very interesting process. Thanks for sharing. The hero of my main project is a statistician. To some it may sound boring, but he a revolutionary swept up by events bigger than he can fathom into an adventure, into space and SPOILER becomes a martyr while potentially saving the course of humanity. (The last part is up in the air because a) he died and b) I want to leave with a cliffhanger and raise more questions than answers in my cautionary tale.) I hated stats in University.

So I take it you select the best of the best to look at and/or share on SaidIt? You've gone an biased me now. I may be looking for your posts more now.

I'll have to look into that IFTTT thing.

I wouldn't want to post all my likes on YouTube. Sometimes I just do them to help the algorithms of indie folks.

I'm not interested in anything Reddit. I didn't like it before SaidIt and I don't like it now, and I despise the censorship and/or SJW cultures.

I like the autopost idea.

I also wonder if I could also make a separate auto-download thing. I would love to just have my channels auto-download to my harddrive as they appear, with the thumbnail, and with my naming scheme - much like my YouTube-DLG does, but without having to find an copy paste new videos within my fave channels.

[–]TheWebOfSlime[S] 2 insightful - 1 fun2 insightful - 0 fun3 insightful - 1 fun -  (2 children)

Off the top of my head, the best way to do that would be to get the link automation down. For example, if there is a channel where you want every video, go to IFTTT, have it import the link, title, etc, to a spreadsheet.

In my experience, there is no reliable downloader, so you probably have to rotate once in a while. There is a constant battle between those type of sites and youtube, itself. But, once you have the link, you can reverse engineer how the download site does its own link... for example (https://www.youtubnow.com/watch/?v=y1GMS4C8S8g)

If you wanted to tackle webhooks in IFTTT, you can then use a second Applet to push the link.

  1. Make Applet When Channel Posts New Video>Create Row in Google Sheets.

  2. Make Applet When New Row Added to Sheet>Webhooks to push the button on the page with generated link.

[–]JasonCarswell 2 insightful - 1 fun2 insightful - 0 fun3 insightful - 1 fun -  (1 child)

I know YouTube is often trying to change things up. They're not very good at it. But it's effective enough to be intermittently annoying. I would rotate and would usually come back to my fave, but the last rotation for a better solution landed me on YouTube-DLG, the best I've found by far, not an addon, but it an open source stand alone. If someone made an addon to bridge the browser and it that could take off. ( /u/wizzwizz4 ? )

I guess I have to learn what Google Sheets is all about too.

Thanks again.

[–]wizzwizz4 2 insightful - 1 fun2 insightful - 0 fun3 insightful - 1 fun -  (0 children)

Sorry; that's REALLY hard. It's a Python program, which means that you'd have to use empythoned to get it to run. Since that's a dead project, you'd have to recompile CPython to LLVM using emscripten yourself. If you can get youtube-dl (the command-line version) running in the browser using empythoned, it should be relatively easy to hook that up to a blob stream and so display it in HTML5, but that'd be inefficient and CPU-eating that's a price we'll have to pay.

Porting it would be easier, but much more time-consuming.

[–]JasonCarswell 2 insightful - 1 fun2 insightful - 0 fun3 insightful - 1 fun -  (2 children)

A++ Very cool. Can you share the code?

[–]TheWebOfSlime[S] 2 insightful - 1 fun2 insightful - 0 fun3 insightful - 1 fun -  (1 child)

Could do. Everything is still pretty rough as I am by no means an expert.

The source of the site, itself, is pretty basic.

The code to get everything to update in real time is still giving me a headache, but I'm almost there.

As for arranging a news article to propagate across multiple platforms, that is more of a group of services and strategies to make that come together.

For example, Instagram, Twitter, Ga.Ai, already have perfectly good features to cross-populate. You can augment those abilities with IFTTT to a great extent and beyond that, you can run scripts inside of Google Sheets. You don't need to know how to code to do any of that. But, once you have a server up, you can run code on the database that should be exporting from sheets.

So, sure, I can share... it just depends what you want.

[–]JasonCarswell 2 insightful - 1 fun2 insightful - 0 fun3 insightful - 1 fun -  (0 children)

Thanks for the breakdown.

Personally I have no use for it, but I was thinking that it would go great on SaidIt if /u/d3rr wanted to implement it somehow.