MEMEX : 'Google search on steroids' (which may or may not be a good thing...)

[Bill Ryan]

.

From http://cbsnews.com/news/new-search-engine-exposes-the-dark-web

<— visit this page for the embedded video news report


~~~~

New search engine exposes the "dark web"
Memex, a powerful new search tool that goes beyond the realm of
Google, Yahoo, and Bing, is launched by DARPA

This week on 60 Minutes, Lesley Stahl and producer Shachar Bar-On got an early look at Memex, a powerful new search engine developed by DARPA (http://www.cbsnews.com/news/darpa-dan-kaufman-internet-security-60-minutes/), the U.S. military's Defense Advanced Research Projects Agency.

The inventor of Memex, Chris White, sat down with Stahl and Bar-On and explained how Memex works--and how it could revolutionize law enforcement investigations. (See Chris White's demonstration in the video player above.)

"The internet is much, much bigger than people think," White said. "By some estimates Google, Microsoft Bing, and Yahoo only give us access to around 5% of the content on the Web." That leaves a lot of room for bad actors to operate freely in the shadows.

White says that Memex goes far beyond the realm of traditional search engines and gives law enforcement a powerful new tool to search the "dark web," where criminals buy, sell, and advertise in the illegal weapons trade and sex trafficking.

"The easiest way to think about Memex is: How can I make the unseen seen?" said Dan Kaufman, director of the information innovation office at DARPA.

"Most people on the internet are doing benign and good things," Kaufman said. "But there are parasites that live on there, and we take away their ability to use the internet against us-- and make the world a better place."

http://cbsnews2.cbsistatic.com/hub/i/r/2015/02/08/8a288236-c5a5-48fa-9333-3dc011ce0923/thumbnail/770x430/2af4f5e173927ca933dd2321018083b4/oodarpa1920.jpg

[Mandala]

A most interesting video, Bill. It seems a great tool for use in "positive" Law Enforcement, but definitely a threat to personal privacy issues. I am certain those with power will no doubt use is as they see fit.

And of course the other thing, is if they are willing to show and publicize this, they are much further along than what has been shown to us. The surveillance rabbit hole is deeper than we can imagine. There is no privacy. We are all watched, recorded, and stored.

[shadowstalker]

A most interesting video, Bill. It seems a great tool for use in "positive" Law Enforcement, but definitely a threat to personal privacy issues. I am certain those with power will no doubt use is as they see fit.

And of course the other thing, is if they are willing to show and publicize this, they are much further along than what has been shown to us. The surveillance rabbit hole is deeper than we can imagine. There is no privacy. We are all watched, recorded, and stored.

I am not surprised at any of this at all saw this coming 5 years ago, but no doubt it is older then that.

[shadowstalker]

I hate getting sudden flashes in my head, what if this is some how connected with or within an A.I. system?

[Tesla_WTC_Solution]

Seems to me that the only fair way to implement MEMEX is through AI.

If they let the "pigs" do it, the enforcement will be one-sided, greed-driven, and will contribute to the development of a very negative mindset.


Giving Memex to the wrong people would be like arming a baboob troupe with AK-47s.

Surely before handing out new weapons to the police, the global community should work on cleaning up the racism in the force! Etc.!

just my two cents.

Also everyone here should considering buying a copy of the most recent "Private Investigator's Little Black Book"; if you are licensed you have some protections.


<3 Bill, remember, there is no such thing as a private forum;
we can hide from each other but apparently not from Big Brother!!

[WhiteLove]

The issue I have with this is that the world does not most of all really need more ability to be controlled, it just needs to be demilitarized by a peace agenda. Weapons of mass destruction and all of the military industrial complexes out there are a far greater threat to humanity. This world simply needs peace and the walk towards peace needs to happen in priority where the threat is the greatest.

To me this Memex is just one more tool by the military to expand dominance, control and reach. In combination with what is missing - demilitarization - it becomes deadly or at the very least very dangerous.

Thanks Bill for bringing up such an important topic!

[Dennis Leahy]

I cannot imagine that there is ANYTHING at all good about this for ordinary citizens. I assume that their mention of "sex trafficking" is a ruse - the top tiers of the pyramid have no qualms about sex trafficking because they are profiting from it and every other nefarious and malevolent action that makes money.

Just as DARPA says that gookle only shows us about 5% of the Internet (so much for "net neutrality" - they have already been restricting our Internet reach by their algorithms) so (I believe) it is true that we do not even know 5% of the Elite's psychopathic depravity.

[Tesla_WTC_Solution]

I cannot imagine that there is ANYTHING at all good about this for ordinary citizens. I assume that their mention of "sex trafficking" is a ruse - the top tiers of the pyramid have no qualms about sex trafficking because they are profiting from it and every other nefarious and malevolent action that makes money.

Just as DARPA says that gookle only shows us about 5% of the Internet (so much for "net neutrality" - they have already been restricting our Internet reach by their algorithms) so (I believe) it is true that we do not even know 5% of the Elite's psychopathic depravity.

I believe that a more modern world will require more morally conscious citizens (my own disgracing is a good example),
but at the same time, a more modern world will exact absolute fairness in the treatment of these citizens,
or collapse!

poof, two minutes to midnight, etc :(


p.s. the scifi writers showed many possible futures.
some are futures where humanity thrives in fair competition/cooperation.
others show a few managing to survive in spite of the system, and happily.
others still show disaster, imprisonment, doom.

which future will humanity choose for itself?
a prison cell or an open sky?

[Ernie Nemeth]

The facts in this story are very much skewed in favor of the establishment, there is no doubt. To me it is interesting that the gatekeepers are that good already at filtering out what the average user can even see on the web. Net neutrality is the farce. Just like the proverbial level-playing field, it not so much went the way of the dodo, nope, it went the way of the flying pink elephant - it never existed at all.

[Tangri]

I cannot imagine that there is ANYTHING at all good about this for ordinary citizens. I assume that their mention of "sex trafficking" is a ruse - the top tiers of the pyramid have no qualms about sex trafficking because they are profiting from it and every other nefarious and malevolent action that makes money.

Just as DARPA says that gookle only shows us about 5% of the Internet (so much for "net neutrality" - they have already been restricting our Internet reach by their algorithms) so (I believe) it is true that we do not even know 5% of the Elite's psychopathic depravity.

If they want to, they can make you (create a new persona) very easy a "sex trafficking person" or a pedophile.

[Mandala]

It's a great way for the "elite pedophile" rings to conduct business. My question is what do you do with the "watchers" who discover things they shouldn't discover. Will they become whistle blowers, or will they have tragic accidents? That's why AI must be the "watcher", because we know if humans do it, there will be one or two along the way that have to morally balance what they have learned and will become the new Snowden.

Compartmentalize times 100.

[Carmody]

This is the way we used to work the net before the net came along. I'm guessing that it is all about address searching, infiltration, private systems, etc... and not about indexed systems.

It's not all that different than a human body. We notice, connect with and live in about 5% of what a human is.

[Alpha141]

The image posted of the relationship model looks a bit like when you install an add on in Firefox Web Browser that allows you to graphically see who is tracking your search history and personal information. It maybe a little off topic but perhaps something to consider for yourself:

https://www.youtube.com/watch?v=2aesvIjQyAo

Very interesting info. Thanks

[christian]

When they say 95% of the web is invisible to major search engines, does that include password restricted areas such as mail accounts and private hangouts? Does Memex try to index that or what?

I'd think that if the NSA reads all mails etc, they also have their own search engine for all of that content already… Google is only for the peasants, I guess.

An open source search engine that really has access to all of the open web content would be great, I think, along with personalized searches and so on…

[Ilie Pandia]

When they say 95% of the web is invisible to major search engines, does that include password restricted areas such as mail accounts and private hangouts? Does Memex try to index that or what?

I'd think that if the NSA reads all mails etc, they also have their own search engine for all of that content already… Google is only for the peasants, I guess.

An open source search engine that really has access to all of the open web content would be great, I think, along with personalized searches and so on…

I personally find these numbers non-reliable. And I did not see anything in the article or the video to convince me otherwise.

What is "visible" on the web depends very much from where you are looking!

Let's take Google.

Google say they index your site (and therefore make it in "visible" in their search results) based on a set of rules. You, as a website developer have the ability to opt-out from Google indexing entirely or just for parts of your website (see the use of "robots.txt" and sitemaps). However, such "opt-ing" out is advisory only. In other words, you can only tell Google: please do not look here, but you have no way to actually enforce that Google follows that rule!

In theory, Google does follow the rules. I am not aware of any blatant violation. But me not being aware, of this not being publicly known does not mean that Google is actually behaving.

Google does read emails. They even say as much. They use that data to serve you what is called "relevant ads". They motivation is: we read ALL email so we don't know which one is actually yours! Again, that is true, in theory, but a motivated third party can create the connections.

Content behind passwords. That is usually protected, unless you use Google Adwords, AdSense, Google Analytics or Google Translate. If you do, then by including that "third party" code from Google, you grant them access to whaterver you are looking at, private or not. Again, I have no evidence that Google has abused this power, but since the possibility of abuse exists, lack of evidence is not proof of non-abuse.

Let's take your ISP.

Things get VERY interesting at this point :). Your ISP sees everything that passes through your network, unless is encrypted. And even encryption will show what sites you are looking at and where you are located, unless you use things such as TOR.

Let's take someone like NSA.

NSA has likely access to a combination of data collection systems from search engines (such as Google), to ISP logs, to actually data on the wires. So they have the ability to pretty much read anything that is not encrypted. Add to this the ability to actually decrypt some stuff (but this is not well documented).

So, where does Memex fit?

It my understanding, Memex sits on top of Google, adding some functionalities and also ignoring any "opt-outs" that some "trafficking sites" may use to keep invisible from Google. It also keeps a history of what is happening (even though Google does that as well with their "cache feature"). So Memex would be able to find temporary pages, making it impossible for someone to delete their online footprint. Even if they delete or change the web pages, Memex would still have a backup of the old data. Another thing that Memex does is to create those "cross-references" that would be nearly impossible to do manually. This is not new. Google does something similar with their Analytics, but it's marketed as "customer targeting and segmentation service" and they explicitly ban sex ads and other offensive stuff. That again does not mean they do not keep track or it or index it, it's just that they don't make it publicly available.

A smart script kid, with enough time on their hands, could use Google Searches to simulate what Memex does. And I suspect that this is pretty much how Memex actually work, by leveraging and creating connections ON TOP of the search engines already existing data, adding to that websites that are usually ignored for various reasons.

I am not sure if I have clarified anything or not :). The thing to take away is that we are being monitored by different parties at different levels and each level has their own searching and indexing tools. It's also likely that all such parties cooperate behind close doors to consolidate data for unstated purposes. What Memex does is not new, and the reference to the "sex trafficking" seems to be the surgar around the bitter pill of accepting more privacy violations in the name of safety. I am sure it has the potential to help close down traffic rings, but what I am not sure of is if it will actually be used for that :). Looking at what Eduard Snowden and others have leaked, such systems are rarely used as advertised, sometimes because if they would be they would cut funds from places where "black budgets" are needed.