Conspiracy search engine

1  2018-08-03 by grijalva10

Since google is clearly censoring “conspiracy” related content and it looks as if this trend will progressively become worse on other search engines, I thought it may be a good idea to develop a search engine exclusively for “conspiracies”. I have 15+ years development experience and think this could really add value to the community and mankind in general. I want to hear your thoughts. If I get enough positive feedback, I will develop a conspiracy focus search engine on github, ideally with the help of this community.

Would this be something you’d like to use?

34 comments

Tor

Tor is a protocol, not a search engine.

Yes I have already developed some applications and hardware utilizing the Tor network

Is dogpile with unsafe search no good?

I'd certainly use something like this and may even offer to help in some way. How do you envision it working? Would it be like a traditional search engine or more like a searchable database? Check out www bibliotecapleyades .net. An updated version of that would be legit!

more like a searchable database?

To me, this would be much more useful. I think the replacement of humans in an editor/curator role with algorithms has been a terrible development in the trajectory of the internet. Algorithms and "machine learning" are great, but only when the supplement human work rather than supplant it.

Wikispooks somewhat serves that role of a searchable database, but leaves a bit to be desired.

I completely agree. I'd envision an updated version of bibliotecapleyades would be gold. A nice GUI and quick search database of all things conspiracy theory related.

The juices are already flowing in my brain on how it would work and function. I'd probably use something like that daily. Wikis are cool and all but have many drawbacks. Hell, we need to give the propagandists at snopes some competition and resistance.

I'd certainly be interested in working on that. Biblioteca's a great resource in terms of content, but is horrible IMO as far as content discovery. It's a library without a card catalog or a knowledgeable librarian.

Wikis are cool and all but have many drawbacks.

Yeah, to me the biggest drawback is that content and curation are too coupled. Just look at how wikipedia got rid of the CIA interventions page, for one example among millions.

Ideally, you'd have a massive repository of content, with editors and teams of editors working to make different "views" of that same content. Completely decouple the content management from the curation.

Yes I had biblio in mind. Searchable database, decentralized node and user curated.

No. It would be targeted hard and filled with noise.

Unless you spend a shitload of time curating it.
And then we'd just be getting your view of what is allowed.

fuck off with your thinking, man. it's a distraction from all the distraction.

And then we'd just be getting your view of what is allowed.

Not necessarily, if done right. Human curators are still better than algorithms, especially if it's truth your after rather than monetization/indoctrination. The trick IMO is to democratize curation, which is where reddit has failed (or from their perspective, succeeded) spectacularly.

On reddit, the user's view of content is determined largely by unelected and unaccountable moderators, many of which are clearly in league with each other. This was due to a conscious decision to place control in the hands of moderators rather than curators per se. You subscribe to subreddits which are controlled static moderators, but I could equally conceive of a system where you subscribe directly to moderators/curators.

Exactly what I was thinking. A user curated independent search engine.

I've spent a bit of time thinking about this and similar ideas (such as a decentralized reddit alternative. I haven't worked out all the details and have had less time for the project as of late, but I think you could make a realistically usable decentralized curation platform on top of reddit's infrastructure through the API, which could be accessed via a browser extension, mobile app, and potentially even a static site that accesses reddit content through the API. That's a bit of a different angle, albeit a similar goal.

For the idea of a user curated search engine, what would be needed would be a minimal viable product (FLOSS product, not commercial) that provides a direct benefit to users they can't find anywhere else. I'm not sure what all would be needed from a technical perspective, but I'd be willing to contribute thoughts and dev time if your serious about making it.

While not required, you are requested to use the NP (No Participation) domain of reddit when crossposting. This helps to protect both your account, and the accounts of other users, from administrative shadowbans. The NP domain can be accessed by replacing the "www" in your reddit link with "np".

I am a bot, and this action was performed automatically. Please contact the moderators of this subreddit if you have any questions or concerns.

no offense intended...

but i'm going to guess you've never run any sort of public forum or other userspace before?

You’re probably spot on. I just think search engines will topic specific moving forward.

Ya. Just look what happened to wikipedia.

Every topic is it's own little feifdom of hitlers on a topic.

Good until it gets political or intersects BIG MONEY. and then...

Users all have agendas.

And you're open to brigading as well.

Hey these 500 people all volunteered to curate our stuff! awesome!

Bob's a great curator. Alice seems like a shill to me. I like the content Bob provides, so I add him as a trusted curator. My results would be customized based on who I have as trusted curators.

Content curation doesn't have to mean that all curators' content goes to a single centralized source of truth, the curators act as filters, which the user chooses, to bring forth the content they want.

There is no longer any technical reason we have to rely on centralized content curation any more, only sociological and commercial reasons.

Bob was faking until he can recommend himself..
Bob reccomends jill whos a bot that bob controls.

repeat until you have a network of plausible deniability that they are ok. but are not.

Unless you do it 100% yourself. And then you'll be flooded out.
Only so many hours in the day...

You can not make technical solutions to social problems.
I know. Sure seems like it's possible. But so far... nope.
technical systems can be gamed.

You can not make technical solutions to social problems.

We need to precisely define the problem if we wish to solve it. As I see it, the problem is:

How do I obtain access to information I deem important to understanding the world?

As a first example, let's consider google. Google of the early 2000s to maybe 2010 was decent solution to this problem. It's algorithms were mostly tuned to finding the content the user wanted to see, and it did better at that goal than pretty much any predecessor. The one caveat was that you needed to know what it was that you wanted to find. After ~2010 or so, Google, needing to monetize its existence, changed the solution it was offering to "how do we sell ads?" (And this is just the surface level, discounting Google's relationship with the US IC).

As another example, consider BBS, forums, etc. These are actually rather powerful, because you don't necessarily need to know what it is your looking for to find it. You find a community of people interested in the things your interested in, and can find the information that community finds interesting. In its golden age, reddit was quite excellent at providing this. For uncontroversial, niche topics, I'd argue it still is (along with well-moderated, single-issue old-school forums).

The problem is that for anything remotely controversial, or where there exists a profit motive somewhere along the line, reddit is easily gamed. The votes are faked, the comments are faked, and the moderators are either bought or ideologically biased. Where the information is not outright silenced, it is sidelined.


Yet I'd argue the solution is a technical means to reproduce a time-tested, even ancient, social solution to a social problem. Trust, but verify. In the days before instantaneous mass communication, where most social networks extended roughly to the distance of a day's journey by that time's technology, people got their information from people they trusted, rightly or wrongly. How did they determine trust? By verifying the information the source produced. "Authoritative" institutions, such as states and religions, often abused this trust, and many if not most were fooled most of the time, but independent thinkers of all ages were able to ingest and analyze information they received and do a sufficient job of approaching the truth of a matter.

Today, the problem is complicated by intermediaries who we, for the most part, do not choose. You did not choose to have me or any of the other mods of this sub to moderate the content you see. You did not choose the various users and bots who vote on the content that makes it to your eyes. You have no realistic way of finding better intermediaries except those already in your social circles, or who have access to the mass markets.

This has been an insoluble problem technically up until the last few years, because most humans could not reach every other human without some centralized intermediary, be it Google or Facebook or Reddit. That's changed now. Most humans, at least in the developing world, have computers in their pockets more powerful than servers of a couple decades ago, and access to as much bandwidth as corporations of 10 years ago. We no longer need to be dependent on intermediaries we don't choose.

hero failure

IPFS could be an interesting solution for the data end, but as of today the infrastructure is pretty immature for developing rich, multi-user applications on top of it.

Possibly decentralized?

Try goodgopher.com

That’s actually pretty good. How could it be better?

In my experience the results are much less intuitive than a normal search engine. Sorta like a years ago engine.

The developers seemed very open when it came out originally. Maybe contact them and build upon their base? Seems like a more efficient method than starting from scratch.

For now, DuckDuckGo works really fine

Down to test

Do it like the original Yahoo did. I.e. there are categories and users can submit links which are then vetted.

Maybe make the vetting crowdsourced? So all links are shown but maybe can be upvoted? (careful with brigading though! don't include downvotes)

This is a good idea

I made a post here on r/Conspiracy What is on Michael Jackson's Tombstone with a decent amount of views and feedback. But you won't find it on a search engine.

I also made the same post "What's on Michael Jackson's Tombstone" on a popular international gaming website forum. The thread was almost immediately locked.

BUT When i search "Michael Jackson Tombstone" in incognito mode on both duckduckgo and yahoo the gaming thread shows up as the first link. I didn't even search the exact title of the thread. Can anyone confirm this and it's not just me?

When I use google in incognito it's pushed back to page 5. It used to be on page one.

Maybe try posting on other popular forum platforms, even if conspiracy topics are banned and locked, your post could reach a wide audience.

Your idea sounds good too.

Someone reccomended https://searx.me yesterday. Apparently their results are non-indexed by Google.