Editing Talk:1909: Digital Resource Lifespan

<!--Please sign your posts with ~~~~ and don't delete this text. New comments should be added at the bottom.-->
Even PDFs can be broken, which is why we have PDF/A (archive) - a subset of PDF that has no external dependencies and thus should last forever.

'''CD scratched, new computer has no CD drive anyway.''' - First, you can still buy external CD-ROM drive, for example connected via USB cable.  Second, you can try recover data from scratched CD with tools such as ddrescue (free and OSS) or IsoBuster (shareware). --[[User:JakubNarebski|JakubNarebski]] ([[User talk:JakubNarebski|talk]]) 17:51, 30 October 2017 (UTC)

: Scratches on the DATA layer of any optical disk destroys that DATA. There is also the consideration that the plastics of the majority of optical disks degrade with time and heat. There are some optical media that are designed to prevent such scratching or corruption like the commercially available M-Disk or laser etching into a micro format into a crystal like a 5D disk. Even then the DATA stored must be in an ISO format to read as well as the equipment to read the media needs to be maintained. I have often told people that their data is never safe unless there is a constant effort to copy, check for quality, and make multiple backups using multiple modern mediums as often as humanly possible. All form of digital media can fail, even the extended warranty on a high end HDD will not cover the data lost and most EULAs for cloud storage will say the same.

::Pressed commercial CD-ROMs carry their information between two 0.6 mm thick plastic discs which are glued together, which makes them pretty resilient against scratches on either side – just remove some material with abrasive methods like toothpaste. Often the glue is the bigger issue with low-quality pressings in the long run. This is in contrast to recordable CDs, which are coated with the reflective layer on top of a single disc. –[[User:TisTheAlmondTavern|TisTheAlmondTavern]] ([[User talk:TisTheAlmondTavern|talk]]) 12:24, 31 October 2017 (UTC)

:Or cheaper than an external drive, borrow a friend's computer and copy the CD onto the cloud somewhere. --[[User:Angel|Angel]] ([[User talk:Angel|talk]]) 18:39, 30 October 2017 (UTC)

: Yet something affected by that would just as likely be affected by "Broken on new OS, not updated". For example, I've got a multimedia encyclopedia which runs on Win 3.11, and thus can't run on 64-bit windows. 
:: Ehrm... You do realise the limitation is the other way around right? You can't run 64-bit application on 32-bit Windows, but 64-bit windows can perfectly well run 32-bit apps. Though Win 3.11 is far enough back it might actually be a fun challenge to see if it runs :D [[Special:Contributions/162.158.202.202|162.158.202.202]] 10:57, 31 October 2017 (UTC)
:::You can not – Win 3.1(.1) was a 16bit operation system – and Microsoft dropped the 16-bit-layer in win7. --[[User:DaB.|DaB.]] ([[User talk:DaB.|talk]]) 19:18, 31 October 2017 (UTC)

Interestingly, static .PDF files are intended to be electronic equivalents of printed books - an electronic microfiche if you will [[User:RIIW - Ponder it|RIIW - Ponder it]] ([[User talk:RIIW - Ponder it|talk]]) 18:57, 30 October 2017 (UTC)

I'm wondering if data on an older, static, website would still be readable. Would likely still be there (or on archive.org), but might be suffering progressive link rot. Also a little surprised that the start of microfilm is so recent; I remember the library having microfilm readers (that nobody ever used) when I was young enough to spend ages staring at a machine, trying to determine its purpose. Guess it depends on the subject, when it was put into that format. --[[User:Angel|Angel]] ([[User talk:Angel|talk]]) 18:39, 30 October 2017 (UTC)

Angel, note both the My in the title and the left arrow implying that the resource (like books) were about before Randal had access. [[User:RIIW - Ponder it|RIIW - Ponder it]] ([[User talk:RIIW - Ponder it|talk]]) 18:57, 30 October 2017 (UTC)

"Only to realize'''d'''? -[[Special:Contributions/172.68.110.22|172.68.110.22]] 23:08, 30 October 2017 (UTC)

[Subject] wiki, anyone? Wikis have rather detailed analyses of even obscure topics in my line of work/study. <sub>--[[User:Nialpxe|<span style="color: #000; text-decoration: none;">Nialpxe</span>]], 2017. [[User_talk:Nialpxe|<span style="color: #000; text-decoration: none;">(Arguments welcome)</span>]]</sub> (P.S. just to be clear I mean wikis maintained by researchers and professionals in [Subject] field, not Wikipedia)

There's a wealth of thought about exactly this problem by librarians;  [https://www.loc.gov/preservation/resources/rfs/TOC.html the Library of Congress has some recommendations] along with [https://www.loc.gov/preservation/digital/formats/index.html a database evaluating over a hundred formats] along a variety of axes: is the format documented openly? Is it widely used? Is it inherently transparent to inspection even if the specification is lost? Can it contain its own metadata? What sort of external dependencies does it have? Is it patent-encumbered, and are there technical access restrictions like DRM? (tl;dr, images as TIFF, text as EPUB or PDF/A, sound as WAV. They're very conservative.) [[Special:Contributions/108.162.249.106|108.162.249.106]] 05:07, 31 October 2017 (UTC)

Note that digital data have big advantage over books when dealing with bigger quantity. The amount of work you need to make to preserve printed book is same no matter how many books you have - so it's thousand times more when you have thousand books. Meanwhile, the amount of work needed to preserve for example collection of digital images doesn't really depend on collection size. Let's say that the used format is going out of use: you can automatically convert all images fairy quickly. Of course, harder with applications ... -- [[User:Hkmaly|Hkmaly]] ([[User talk:Hkmaly|talk]]) 08:23, 31 October 2017 (UTC)

The software not running after OS update is such a Mac problem. Linux updates would break if closed software was commonly available, but open source can be recompiled, and Windows maintain a scarry amount of backwards compatibility, and only system-admin or DRM-crippled software ever stops working.[[Special:Contributions/162.158.202.202|162.158.202.202]] 10:54, 31 October 2017 (UTC)

Here in the UK, the library access would also have ended some time in the last few years...[[Special:Contributions/162.158.155.32|162.158.155.32]] 11:33, 31 October 2017 (UTC)

Nothing lasts forever (or at least that's what seems to be true for anything observed by humanity). Data becomes corrupted and lost over time and usage, and books become damaged and lost over time and usage. Not to mention, thousands of books were burned during the Nazi regime. Human minds are inevitably subject to corrupted memories as well. We lose information all the time, and we try to recover what remains. However, it is also worth mentioning that our digital technology is still pretty young compared to books and other sources of information. Information used to be recorded on papyrus, tablets (I understand that this contradicts my point as some tablets have stood the test of time), etc. Some of the earliest Chinese inks were created with soot and animal glue. The first (attempts of) photographs required hours of light exposure and would fade away quickly. Over time, we discovered ways to improve upon these sources of information. The same could apply to our digital information today. We are essentially in the "papyrus" phase of electronic technology (one could argue with other descriptions, but this isn't significant to my statements). In time, we may achieve more successful long-term solutions to maintaining original data. There are so many avenues for the advancement of technology, and those avenues continue to multiply with each step. At this time, we just need to continue to work on our projects and experiments for the progress of humanity. [[User:NAE|NAE]] ([[User talk:NAE|talk]]) 14:29, 31 October 2017 (UTC)

Randall did a good job frightening me this Halloween... [[Special:Contributions/172.68.34.88|172.68.34.88]] 02:10, 1 November 2017 (UTC)

I wonder if Randall is aware of digital archiving solutions such as those provided by Preservica (https://preservica.com/), formerly part of Tessella plc. Their solutions are aimed at precisely this problem. Their library/museum clients include "the MoMA, the Frick Collection, the Museum of Fine Arts Houston, Yale Library, The National Library of Australia, The Royal Danish Library, The Philadelphia Museum of Art, McNay Art Museum, DC Public Library and the University of Manchester" and their archive clients include "15 leading pan-national and national archives, 18 US state archives, major corporate archives at BT, HSBC, Unilever and the Associated Press". [[Special:Contributions/108.162.249.88|108.162.249.88]] 03:32, 1 November 2017 (UTC)
@@ Line 1: / Line 1: @@
 <!--Please sign your posts with ~~~~ and don't delete this text. New comments should be added at the bottom.-->
 Even PDFs can be broken, which is why we have PDF/A (archive) - a subset of PDF that has no external dependencies and thus should last forever.
-[[User:JakubNarebski|JakubNarebski]]
-:To clarify: .PDF files are *frequently* created with content such as fonts (or really anything other than the actual text) referenced within the document but not *embedded* within the document. This is usually done to reduce file size, but it's usually not advisable. Whether it's a .pdf or a .ppt or a .exe it is best to keep your dependencies embedded whenever possible!
-:.PDF files (or any files) can of course also suffer from hash failure (CRC errors, etc) and PDF/A does not provide redundancy tables; Always make an extra copy on another drive (ideally both off-site & locally).
-[[Special:Contributions/162.158.69.77|162.158.69.77]] 06:07, 1 November 2017 (UTC)
 '''CD scratched, new computer has no CD drive anyway.''' - First, you can still buy external CD-ROM drive, for example connected via USB cable.  Second, you can try recover data from scratched CD with tools such as ddrescue (free and OSS) or IsoBuster (shareware). --[[User:JakubNarebski|JakubNarebski]] ([[User talk:JakubNarebski|talk]]) 17:51, 30 October 2017 (UTC)
@@ Line 11: / Line 7: @@
 ::Pressed commercial CD-ROMs carry their information between two 0.6 mm thick plastic discs which are glued together, which makes them pretty resilient against scratches on either side – just remove some material with abrasive methods like toothpaste. Often the glue is the bigger issue with low-quality pressings in the long run. This is in contrast to recordable CDs, which are coated with the reflective layer on top of a single disc. –[[User:TisTheAlmondTavern|TisTheAlmondTavern]] ([[User talk:TisTheAlmondTavern|talk]]) 12:24, 31 October 2017 (UTC)
-::: CD-ROMS *always* carry their information directly below the reflective layer on the "printed" side of the 1.2mm disk (so around 0.1mm below the print). That's necessary because otherwise the laser would not correctly focus on the data. The recordable CDs have the problem that the reflective layer may not be 100% air resistent and so oxydise the data layer, which is a dye. It's DVDs that have two 0.6mm disks, so in theory you cold flip the DVDs like an LP and use both sides - but then you don't have a surface to print information on it (except the few square cm just around the hole). BlueRay Disks inverted the CD: now the data layer is behind a 0.1mm "thick" coating on the data side. -- [[Special:Contributions/162.158.88.74|162.158.88.74]] 11:23, 8 November 2017 (UTC)
 :Or cheaper than an external drive, borrow a friend's computer and copy the CD onto the cloud somewhere. --[[User:Angel|Angel]] ([[User talk:Angel|talk]]) 18:39, 30 October 2017 (UTC)
-::What if you don't have any friends? (or what if none of your friends has a CD drive) --[[Special:Contributions/108.162.216.40|108.162.216.40]]
-:::You can still buy external friends that have CD drives.--[[Special:Contributions/162.158.89.61|162.158.89.61]] 13:12, 3 November 2017 (UTC)
 : Yet something affected by that would just as likely be affected by "Broken on new OS, not updated". For example, I've got a multimedia encyclopedia which runs on Win 3.11, and thus can't run on 64-bit windows.
 :: Ehrm... You do realise the limitation is the other way around right? You can't run 64-bit application on 32-bit Windows, but 64-bit windows can perfectly well run 32-bit apps. Though Win 3.11 is far enough back it might actually be a fun challenge to see if it runs :D [[Special:Contributions/162.158.202.202|162.158.202.202]] 10:57, 31 October 2017 (UTC)
 :::You can not – Win 3.1(.1) was a 16bit operation system – and Microsoft dropped the 16-bit-layer in win7. --[[User:DaB.|DaB.]] ([[User talk:DaB.|talk]]) 19:18, 31 October 2017 (UTC)
-::::Most obsolete software can be quite easily run using various emulators or virtual machines. A lot of Win3.1 software runs without problems on modern Linuxes via Wine, and if it doesn't, there are always emulators such as DOSBox - copies of Win3.1 can be easily found on various abandonware sites and [https://archive.org/details/msdos_win3_1 even archive.org] (even though their legality may be questioned). [[Special:Contributions/162.158.202.106|162.158.202.106]] 22:14, 11 November 2017 (UTC)
 Interestingly, static .PDF files are intended to be electronic equivalents of printed books - an electronic microfiche if you will [[User:RIIW - Ponder it|RIIW - Ponder it]] ([[User talk:RIIW - Ponder it|talk]]) 18:57, 30 October 2017 (UTC)
 I'm wondering if data on an older, static, website would still be readable. Would likely still be there (or on archive.org), but might be suffering progressive link rot. Also a little surprised that the start of microfilm is so recent; I remember the library having microfilm readers (that nobody ever used) when I was young enough to spend ages staring at a machine, trying to determine its purpose. Guess it depends on the subject, when it was put into that format. --[[User:Angel|Angel]] ([[User talk:Angel|talk]]) 18:39, 30 October 2017 (UTC)
-: archive.org returns contemporary pages for links on archived pages, so that should be still safe. The worst nightmare with archive.org is a newly-set robots.txt file: Wayback Machine will just pretend to know nothing about the page even if it has been archived in the past. It sometimes crawl pages, after all. --[[Special:Contributions/172.68.54.16|172.68.54.16]] 07:22, 3 November 2017 (UTC)
 Angel, note both the My in the title and the left arrow implying that the resource (like books) were about before Randal had access. [[User:RIIW - Ponder it|RIIW - Ponder it]] ([[User talk:RIIW - Ponder it|talk]]) 18:57, 30 October 2017 (UTC)
-: Should those white left arrows be noted in the transcript?  The gray right arrows are implied by "past", perhaps something like "Before 1980-past 2020"[[Special:Contributions/162.158.63.16|162.158.63.16]] 17:39, 1 November 2017 (UTC)
-"Only to realize'''d'''"? -[[Special:Contributions/172.68.110.22|172.68.110.22]] 23:08, 30 October 2017 (UTC)
+"Only to realize'''d'''? -[[Special:Contributions/172.68.110.22|172.68.110.22]] 23:08, 30 October 2017 (UTC)
-:This is probably a typo'''ed'''.--[[Special:Contributions/162.158.89.61|162.158.89.61]] 13:19, 3 November 2017 (UTC)
 [Subject] wiki, anyone? Wikis have rather detailed analyses of even obscure topics in my line of work/study. <sub>--[[User:Nialpxe|<span style="color: #000; text-decoration: none;">Nialpxe</span>]], 2017. [[User_talk:Nialpxe|<span style="color: #000; text-decoration: none;">(Arguments welcome)</span>]]</sub> (P.S. just to be clear I mean wikis maintained by researchers and professionals in [Subject] field, not Wikipedia)
@@ Line 44: / Line 29: @@
 The software not running after OS update is such a Mac problem. Linux updates would break if closed software was commonly available, but open source can be recompiled, and Windows maintain a scarry amount of backwards compatibility, and only system-admin or DRM-crippled software ever stops working.[[Special:Contributions/162.158.202.202|162.158.202.202]] 10:54, 31 October 2017 (UTC)
-:I must strongly disagree there; Networking features have been known to break following Windows updates, & Android is *terribly* prone to breaking apps or even removing what may be considered core system features with an OS update. Search "kitkat sd", for just one good example. Even Linux can turn into dependency hell when repositories change their branch structure. Then there's the incredible variety of different hardware which only a specific version of Windows with specific hardware once supported: I still can't get an affordable analog serial port adapter that will work with my favorite flight controller.[[Special:Contributions/162.158.75.10|162.158.75.10]] 06:37, 1 November 2017 (UTC)
 Here in the UK, the library access would also have ended some time in the last few years...[[Special:Contributions/162.158.155.32|162.158.155.32]] 11:33, 31 October 2017 (UTC)
@@ Line 53: / Line 37: @@
 I wonder if Randall is aware of digital archiving solutions such as those provided by Preservica (https://preservica.com/), formerly part of Tessella plc. Their solutions are aimed at precisely this problem. Their library/museum clients include "the MoMA, the Frick Collection, the Museum of Fine Arts Houston, Yale Library, The National Library of Australia, The Royal Danish Library, The Philadelphia Museum of Art, McNay Art Museum, DC Public Library and the University of Manchester" and their archive clients include "15 leading pan-national and national archives, 18 US state archives, major corporate archives at BT, HSBC, Unilever and the Associated Press". [[Special:Contributions/108.162.249.88|108.162.249.88]] 03:32, 1 November 2017 (UTC)
-Randall forgot my personal favorite: UTF-8 formatted .txt files. Since 1993 & counting, never had an issue opening one. I still have my first copy of The Anarchist's Cookbook, copied from a Kaypro II running CP/M on a 5-1/4" floppy to an 8088XT running MS-DOS on a 30mb hard drive to an IBM PS/2 286 on 20mb hard drive to an Asus 486 on a 3.5" floppy to a 1.2gHz Pentium on a 100mb Zip drive to a Core 2 Duo on a CD-R to an i7 system on a 128gb solid state drive, which was finally backed up to a 1tb hard drive & archived, as there's a newer copy to carry around. That original file still opens just fine on any PC I've ever used (including mobile).
-: I love TXT files. They're great for reading without the need for pictures and formatting. I also occasionally prefer writing TXT files instead of creating DOC or DOCX files because there are less interface distractions. [[User:NAE|NAE]] ([[User talk:NAE|talk]]) 18:25, 10 November 2017 (UTC)
-Also, I believe Linus Torvalds once said (talking about code, but it applies to anything sufficiently desirable) "Only wimps use tape backup, real men just upload their important stuff on ftp, and let the rest of the world mirror it ;)" I can certainly attest to that. I once made a torrent of all the Star Trek I'd accumulated (IE, all the Star Trek ever) & uploaded that. Two years later an old hard drive died & I was able to recover all 200+ gb in a little over 6 hours, simply by downloading my own torrent from other seeds. Thanks Trekkies![[Special:Contributions/162.158.75.10|162.158.75.10]] 07:22, 1 November 2017 (UTC)
-The transcripts for the books and microfilm should state the date range as "before 1980's" to represent the arrows on the chart, just as "past ..." is used for the bars that have arrows at their right ends. I wonder if there is any significance to the fact that the arrows are done differently for "before" and "after".[[Special:Contributions/162.158.126.28|162.158.126.28]] 14:58, 7 November 2017 (UTC)
-The most reliable form of long-term backup may very well be HD-Rosetta. Which is like microfilm except etched into non-reactive metal. Unfortunately it's super-duper expensive and only a very limited number of capable of minting them even exist. On the upside they're immune to data format changes because they can be read with an ordinary light microscope [[Special:Contributions/173.245.54.123|173.245.54.123]] 02:34, 16 March 2020 (UTC)