Problem: how to safely preserve data and knowledge in digital format?

[Ilie Pandia]

OK,

So I do have backups on local drive as well as on the cloud. Some of Paul's obsessiveness with backups has rubbed onto me as well . But unfortunately I doubt many will have those in place, especially considering how bad the Windows backups programs are.

However I still need another backup plan that will survive any potential "electronics" collapse.

Reading the thread so far it seems that I need to do a few things:

- take the time to look trough and collect into one place all the books and documents that I consider relevant.
- get some Verbatim Gold Archival DVD-R disks and burn the digital documents I've found previously.

To take this one step further it would be nice to have a sort of centralized and indexed library of stuff that many other users could easily copy and duplicate on their machine.

@Paul:
The problem I see with having information so widely distributed is that it may not be easily accessible. With a centralized library thing everybody would know where to look for info, but without a centralized index it would be virtually impossible to know who has what documents. I guess I'm concerned with my over reliance on Google to find the information I require quickly and reliably. But if Google goes away or decides to filter down access to the information I can find that would be a major pain for me.

[Ilie Pandia]

Here are some pieces of software that may come in handy:

Free Document Management Software: http://www.opendocman.com/

Free Tag Based File Management Software: http://silkwoodsoftware.com/index.html (so you can easily locate your files by tags and not having to remember the actual location on the files tree on your hard drive)

[Robert J. Niewiadomski]

Do we have few issues at hand here?
1) way to preserve information so it is 100% lose-less. Carefully choosing recording medium. Making data redundant. Self replicating copies? Like viruses or malware?
2) way to track the changes made to that information so tampering can be detected. Hash codes? Change logs?
3) way to disperse that information and make it redundant. Some kind of distributed file system. Cloud? P2P networks?

If you look "distributed knowledge management" in Google some >3'000'000 entries will appear. But i think Illie the issue you bring forward is a tip of an iceberg. Unless our software and hardware is 100% fault tolerant at least by making it more redundant (i believe it can be done) we are bound for a major catastrophe. Imagine Google or WikiPedia (or the likes) being attacked by malware and their data stores being permanently erased by it. Try to do any "project" without consulting the internet It is possible but it's "time footprint" grows big imo.

The data inside of a hard drives is safe against EMP from the Sun or a weapon. It its recorded onto metal disks surrounded by metal casing of a hard drive. To me it looks like a Faraday cage and should protect it from EMP. But not the electronic components. Fortunately electronics redundancy is unlimited. If it do not contains data. Hard drives are not immune to heat. Heat can alter data by reorienting magnetic domains on the surface of metal disks. And HD's are prone to G-forces acting on it. Accidental dropping can cause moveable parts for reading/writing (heads) to "jump" and scratch metal disks erasing part of the information. You can bury it in sand and it should be ok. Water would also be no problem - except for the electronics.

Optical media (CD, DVD, B-ray, HDisk) is safe from EMP, G-forces and water but not from heat and rough material (ex. sand).

Flash memory is safe from G-forces and rough materials but not from heat, EMP and water.

One drawback of all of them is dependence on electricity and some hardware (failure prone) interfacing that information to humans. Humans can not directly read it So we need mechanical intermediary.

The ideal "data center" would need to combine different media storage devices and make them store redundant information. There is also energy issue. If FE would become unshackled it is obvious it would be used as a power source Until then the data center would need different power sources (ie. wind, photovoltaics, thermovoltaics, fuel-cells, internal combustion, chemical) combined together and made redundant to backup each other. Now the data centers would need to be networked. They could be simultaneusly act as DNS servers, wifi acces points or GSM towers to provide remote access to the stored data inside. We equip them with GPS and place them in standard freight containers form factor to allow for portability.

The data centers would need an operating system and protocol for consolidating distributed information in one service...

[Ilie Pandia]

Posted by Robert J. Niewiadomski (here)
But i think Illie the issue you bring forward is a tip of an iceberg.

Yes, indeed. It seems like that the more I think of it.

The way in which we now handle information, always asking "what's new", quickly forgetting "what's best" and our past mistakes, hopping than in the digital world we can do or say anything because is "not real" so we can easily "erase it", it's a symptom of a much deeper problem.

Perhaps we are indeed overdue for a reboot

[ThePythonicCow]

Posted by Ilie Pandia (here)
So this is a serious question: how do we, as a people, preserve our history, our knowledge and data, from being destroyed, or centralized in the hands of very few that will be able to alter it or grant access to as they see fit?

The first several minutes of the following youtube video present another effort at durable data storage, called "stonebooks", described by Clif High (of HalfPastHuman):

Source: https://youtube.com/watch?v=MIExY-F8T2E

Webbot Project - Clifs Wujo with Clif High E-21 10/14/2012 (full)
E21 - October 14, 2012 - stonebooks update

[Anchor]

Posted by Paul (here)

Posted by Paul (here)
CD's and DVD's aren't "scratches" on a surface ... they are chemical reactions. As best as I recall, the archival life of CD's and DVD's typically varies in the range of about 2 to 15 years, depending on material.

Correction - here's some DVD media claiming to be good for 100 years: Verbatim UltraLife™ Gold Archival Grade DVD-R. Purchasing 100 of them would cost about $160, or about $340 for one Terabyte.

Thanks for this link.

I like your idea of getting bigger and bigger disks and then keeping multiple backups - preferably in different physical locations.

This works because it is maintained and refreshed by a real live human. I dont rate the archival capabilities of my hard disks if say someone from the future dug one up and tried to work out what was on it

[ThePythonicCow]

Posted by Anchor (here)
I like your idea of getting bigger and bigger disks and then keeping multiple backups - preferably in different physical locations.

It worked well for the last 25 years.

I don't expect it will continue to work as well. The exorbitant privilege of earning a wage denominated in the world's reserve currency has allowed me to purchase many disks cheap. The exponential increase in disk data density and size for the last 25 years is also unlikely to continue, due to some coming economic break downs and a reversal of globalization.

I have switched, over the last couple of years, to getting redundant copies of the most durable computer hardware I can afford, with the expectation that I may be using much the same computer hardware for a long time to come.

[ThePythonicCow]

Posted by Paul (here)
The first several minutes of the following youtube video present another effort at durable data storage, called "stonebooks", described by Clif High (of HalfPastHuman):

Here's another explanation of "stonebooks" from Clif High, at StoneBooks Works
Global Challenge...:

September 21, 2012

Presently there are two potentially viable processes for producing our StoneBooks.

We are putting together a plan for crowdsourcing the necessary capital to begin testing the production issues involved with one of these processes. This first test is designed to produce a 'volume' that is 100 pages that are bound in a titanium housing. The process should produce a volume that will last about 30,000 years*.

It will be expensive to produce these so all our funds will go toward that goal. We intend to sell these to institutions, organizations, and rich individuals (we will likely offer 'vanity' versions as well). Any returns will be plowed back into the next set of volumes.

Here is the planetary challenge. We need to have smart people help us by reducing knowledge down to 100 page primers. These primers will have to fit on 5 by 7 inch pages. These primers will have to assume that NO language will be used. These primers need to convey their information in graphic format.

Our first test productions will be for Maths (geometry, and functional building maths), and Biology (specifically human birthing process).

We will be producing volumes on Astronomy (helical model), Biology (all forms of common medical/dental procedures), Biology ( animal husbandry), Farming, and anything and everything else we can think of over these next months.

However, each and every volume is expensive to produce, therefore this will not be a speedy growth industry.

More at the above link.

[Robert J. Niewiadomski]

Paul, i like two things about the idea of Clif's stonebooks. Durability and using graphical form and nothing beside it to describe our knowledge accumulated so far.
But...
1) who will decide how this information will be presented? Will it be biased somehow?
2) the books are expensive to make and there will be not enough copies for everybody when TSHTF They have scarcity build-in in them
3) because of no. 2 people will kill each other to literally hoard that knowledge. If you watch Revolution you will know what i mean...

[rgray222]

I don't think it really matters and in some respect it might be for the better. I know that might seem shocking to some but when you ponder information and how to store it, one of a kind art, priceless manuscripts it seems meaningless in the greater scheme of things. Is there anything worth saving, maybe medical teachings and techniques to extend life but I am not even sure about that. Certainly life is worth saving but even that only last for the blink of an eye so life saving measures are only temporary at best. Death of information, an object or a person is inevitable, I don't really think any of it can be stopped, it can only be delayed.
I think when we get to the place that most of us are striving to reach, that knowledge is 'stored' in a way that it can never be erased or forgotten! The level of man's consciousnesses is on a road that has a final destination, getting there is nothing more than a learning or wisdom gaining experience. Storing information is tantamount to storing a photograph in a box for 20 or 30 years, when you open the box after all that time the image is no longer but the experience you were having when the photograph was taken is forever!

[TargeT]

Posted by rgray222 (here)
I don't think it really matters and in some respect it might be for the better. I know that might seem shocking to some but when you ponder information and how to store it, one of a kind art, priceless manuscripts it seems meaningless in the greater scheme of things. Is there anything worth saving, maybe medical teachings and techniques to extend life but I am not even sure about that. Certainly life is worth saving but even that only last for the blink of an eye so life saving measures are only temporary at best. Death of information, an object or a person is inevitable, I don't really think any of it can be stopped, it can only be delayed.
I think when we get to the place that most of us are striving to reach, that knowledge is 'stored' in a way that it can never be erased or forgotten! The level of man's consciousnesses is on a road that has a final destination, getting there is nothing more than a learning or wisdom gaining experience. Storing information is tantamount to storing a photograph in a box for 20 or 30 years, when you open the box after all that time the image is no longer but the experience you were having when the photograph was taken is forever!

I think "death of information" has happened, but only for "most" of us...a select few understand fundamental reality truths that the rest of us have been lead astray from (for example : our current physics are based on E=MC^2, which is a COMPLETE JOKE, gravity travels FAR faster than the speed of light.. our sun is an electoral focal point not a "nuclear furnace" etc etc etc...)

I'll assume this is the motivation behind this thread... to prevent this from happening again.

[Ilie Pandia]

I just had a thought last night, right before falling asleep

The thought was: "Hoarding information and knowledge is not that much different than hoarding any other kind of stuff, and it's based in fear of loss".

[ThePythonicCow]

Posted by Ilie Pandia (here)
The thought was: "Hoarding information and knowledge is not that much different than hoarding any other kind of stuff, and it's based in fear of loss".

That's the key thought behind Open Source software .

[TargeT]

Posted by Ilie Pandia (here)
I just had a thought last night, right before falling asleep

The thought was: "Hoarding information and knowledge is not that much different than hoarding any other kind of stuff, and it's based in fear of loss".

or anticipation of influence / power?

surely (if what we assume to be "TPTB" are as advanced as it seems they are/ could be) the knowledge was not used out of fear, but out of manipulation & for a power move.

if it was known that a "global reset" was cyclic (perhaps not even cyclic, perhaps just predicted) & it was planned for, those that kept the knowledge would be in a powerful position compared to those that did not know & /or plan for it.

a lot of assumptions there, but not too far off base I think.

[Robert J. Niewiadomski]

Posted by Ilie Pandia (here)
I just had a thought last night, right before falling asleep

The thought was: "Hoarding information and knowledge is not that much different than hoarding any other kind of stuff, and it's based in fear of loss".

Illie, there is nothing wrong with that "hoarding" unless the hoard is not shared and freelly available to others...

I am currently through another book by Joseph Chilton Pearce. First was Magical Child (a must read for all parents! especially those parents planning on inviteing a new human being). Second was The Crack In The Cosmic Egg. Dealing mostly with our perceptionof reality and our apparent misunderstanding of cause and effect issue. You will never look the same after reading those two books.

Back to topic. Third book titled Evolution's End elaborate on topic of reality perception AND the source of knowledge. It is postulated there,backed with real life examples, that human (and probabably non human too) knowledge is stored in non localized field accessible to all. Acces is read/write mode Every being contributes to this field and can freely tap in to it. OBE, RV, telephaty and other simmilar phenomena can be explained by it. It has various names: Akashic records, morfogenetic fields and many other i guess but can't recall... So maybe we should focus on learning to consciously tap in to that already present store than to worry how to backup it?

[rgray222]

I think "death of information" has happened, but only for "most" of us...a select few understand fundamental reality truths that the rest of us have been lead astray from (for example : our current physics are based on E=MC^2, which is a COMPLETE JOKE, gravity travels FAR faster than the speed of light.. our sun is an electoral focal point not a "nuclear furnace" etc etc etc...)

I'll assume this is the motivation behind this thread... to prevent this from happening again.

There is no question that the speed limit that Einstein imposed on the universe is not correct. We also know that mankind will eventually be able to control or manipulate gravity and once that happens on a sustained basis then all bets are off. We also know that black holes greatly distort gravity, that fact alone discounts much of what we believe to be true on a scientific basis. Also quantum physics has been around since the 20's and we have not even scratched the surface, each year goes by the belief in dimensions and multiverses becomes a bit more of a reality.

The thought was: "Hoarding information and knowledge is not that much different than hoarding any other kind of stuff, and it's based in fear of loss".

In a way that was very much my point in the above post (#30). Fear motivates us to do virtually everything and once you understand that you can let a lot of it go. It makes little difference if it is fear of, bad work performance, marriage, loss of wealth, poverty, sickness, weight loss, divorce and yes even fear of losing information or fear of death.