2739: Data Quality - Revision history

2A02:C7C:6D8A:6800:2DFB:1812:23D0:3B8: True colour is 24-bit, not 16, and GIF has always used 1 byte for red, 1 for green, 1 for blue in the global colour map

2026-01-04T11:34:02Z

True colour is 24-bit, not 16, and GIF has always used 1 byte for red, 1 for green, 1 for blue in the global colour map

Firestar233: I did a big rewrite. MPEG contains audio-only formats as well (mp3 for example). Also, i'm not sure he meant hashing of only books, because it applies to all types of data

2025-05-04T03:58:36Z

I did a big rewrite. MPEG contains audio-only formats as well (mp3 for example). Also, i'm not sure he meant hashing of only books, because it applies to all types of data

162.158.146.89: /* Details */

2024-02-17T18:29:44Z

‎Details

172.70.207.141: /* Details */

2024-02-01T23:01:18Z

‎Details

172.70.207.142: /* Details */

2024-02-01T23:00:44Z

‎Details

172.69.33.63: Integrating the title text into the chart

2024-02-01T22:07:39Z

Integrating the title text into the chart

Xurkitree10: /* Explanation */

2023-03-29T17:54:15Z

‎Explanation

172.70.86.127: Undo revision 306844 by NiceGuy1 (talk) Edit reason dors not square with the only hint of any sequence at all (and that being circumstantial!) being the arrow from left to right. Possible. Keep.

2023-02-25T10:08:54Z

Undo revision 306844 by NiceGuy1 (talk) Edit reason dors not square with the only hint of any sequence at all (and that being circumstantial!) being the arrow from left to right. Possible. Keep.

NiceGuy1: /* Details */ No he didn't. He's implying starting on the right and subsequently re-saving it / transferring it in each format from right to left. Saving a JPG as a GIF will indeed lose quality.

2023-02-25T06:54:38Z

‎Details: No he didn't. He's implying starting on the right and subsequently re-saving it / transferring it in each format from right to left. Saving a JPG as a GIF will indeed lose quality.

NiceGuy1: /* Transcript */ As worded it was ambiguous, sounds like JUST beneath instead of far, while it actually means direction.

2023-02-25T06:32:23Z

‎Transcript: As worded it was ambiguous, sounds like JUST beneath instead of far, while it actually means direction.

@@ Line 40: / Line 40: @@
 | a drawing of your cat
 | Image, audio, and video formats that are considered '{{w|Lossy compression|lossy}}'. The JPG (or "JPEG") picture format and the {{w|MPEG}} group of audio and video formats ({{w|MP1}}, {{w|MPEG-1 Audio Layer II|MP2}}, {{w|MP3}}, {{w|MP4}}, {{w|Advanced Audio Coding|AAC}}) typically use a range of data-compression methods that save space by selectively fudging (thus losing) what details it can of the image and audio, to make disproportionate gains in compression ratios; best used for real world images (and films) where real-world 'noise' can afford to be replaced by a more compressible version, without too much obvious change.
-GIF compression is not 'lossy' in the same way, i.e. whatever it is asked to encode can be faithfully decoded, but Randall may consider its limitations (it can only write images of 256 unique hues, albeit that these can come from anywhere across the whole 65,536 "True color" range, plus transparency) to be a form of loss, as conversion from a more sophisticated format (e.g. PNG, below) could lose many of the subtle shades of the original and produce an inferior image. For this reason, GIF format becomes one best left to render diagrams and other computer-generated imagery with swathes of identical pixels and mostly sharp edges (and to utilize the optional transparent mask), for which JPEG compression will create prominent image artefacts. Alternatively, he may just have included it as a joke/nerd-snipe.
+GIF compression is not 'lossy' in the same way, i.e. whatever it is asked to encode can be faithfully decoded, but Randall may consider its limitations (it can only write images of 256 unique hues, albeit that these can come from anywhere across the whole 24-bit "True color" range, plus transparency) to be a form of loss, as conversion from a more sophisticated format (e.g. PNG, below) could lose many of the subtle shades of the original and produce an inferior image. For this reason, GIF format becomes one best left to render diagrams and other computer-generated imagery with swathes of identical pixels and mostly sharp edges (and to utilize the optional transparent mask), for which JPEG compression will create prominent image artefacts. Alternatively, he may just have included it as a joke/nerd-snipe.
 |-
 | {{w|PNG}}, {{w|ZIP (file format)|ZIP}}, {{w|TIFF}}, {{w|WAV}}, raw data

@@ Line 35: / Line 35: @@
 | {{w|Hash table}}
 | your cat's full chip ID
-| A hash table allows you to find data very fast. Randall probably means hashing the contents of entire books. Calculating a hash value for an entire book means that there is (most probably) a unique relationship between the book and a hash value - e.g. "58b8893b172d00e9". This means this exact version of the book will yield this exact hash value, though it's practically impossible to reconstruct the book's potential content from a hash value. It is a method of checking that a copy is the same as the original, but is meaningless on its own and has the possibility of being wrong. An average book contains several millions of bits, yet the SHA-2 hash has only 256 bits, so there are theoretically many (mostly nonsensical, but not necessarily) 'wrong' versions that might look correct.
+| A hash table allows you to find data very fast. Calculating a hash value for a piece of data means that there is (most probably) a unique relationship between the data and a hash value - e.g. "58b8893b172d00e9". This means this exact version of the data will yield this exact hash value, though it's practically impossible to reconstruct the data from a hash value. It is a method of checking that a copy is the same as the original, but is meaningless on its own and has the possibility of being wrong. For example, an average book contains several millions of bits, yet the SHA-2 hash has only 256 bits, so there are theoretically many (mostly nonsensical, but not necessarily) 'wrong' books that might look correct under a hash function.
 |-
 | {{w|JPEG|JPG}}, {{w|GIF}}, {{w|MPEG-1|MPEG}}
 | a drawing of your cat
-| Image and video formats that are considered 'lossy'. JPG (or "JPEG") format and the MPEG {{w|MPEG-2|group}} {{w|Advanced Video Coding|of}} formats typically use a range of data-compression methods that save space by selectively fudging (thus losing) what details it can of the image (and audio, where appropriate), to make disproportionate gains in compression; best used for real world images (and films) where real-world 'noise' can afford to be replaced by a more compressible version, without too much obvious change.
+| Image, audio, and video formats that are considered '{{w|Lossy compression|lossy}}'. The JPG (or "JPEG") picture format and the {{w|MPEG}} group of audio and video formats ({{w|MP1}}, {{w|MPEG-1 Audio Layer II|MP2}}, {{w|MP3}}, {{w|MP4}}, {{w|Advanced Audio Coding|AAC}}) typically use a range of data-compression methods that save space by selectively fudging (thus losing) what details it can of the image and audio, to make disproportionate gains in compression ratios; best used for real world images (and films) where real-world 'noise' can afford to be replaced by a more compressible version, without too much obvious change.
-GIF compression is not 'lossy' in the same way, i.e. whatever it is asked to encode can be faithfully decoded, but Randall may consider its limitations (it can only write images of 256 unique hues, albeit that these can come from anywhere across the whole 65,536 "True color" range, plus transparency) to be a form of loss, as conversion from a more sophisticated format (e.g. PNG, below) could lose many of the subtle shades of the original and produce an inferior image. For this reason, GIF format becomes one best left to render diagrams and other computer-generated imagery with swathes of identical pixels and mostly sharp edges (and to utilize the optional transparent mask), for which JPEG compression will create prominant image artefacts. Alternatively, he may just have included it as a joke/nerd-snipe.
+GIF compression is not 'lossy' in the same way, i.e. whatever it is asked to encode can be faithfully decoded, but Randall may consider its limitations (it can only write images of 256 unique hues, albeit that these can come from anywhere across the whole 65,536 "True color" range, plus transparency) to be a form of loss, as conversion from a more sophisticated format (e.g. PNG, below) could lose many of the subtle shades of the original and produce an inferior image. For this reason, GIF format becomes one best left to render diagrams and other computer-generated imagery with swathes of identical pixels and mostly sharp edges (and to utilize the optional transparent mask), for which JPEG compression will create prominent image artefacts. Alternatively, he may just have included it as a joke/nerd-snipe.
 |-
 | {{w|PNG}}, {{w|ZIP (file format)|ZIP}}, {{w|TIFF}}, {{w|WAV}}, raw data
 | photo of your cat
-| A series of formats using lossless compression. PNG and TIFF are image formats that are suitable for photos, but without (necessarily) resorting to reduced accuracy in order to assist compression. WAV is an audio format that also does not arbitrarily sacrifice 'unnecessary' details, unlike the more recently developed {{w|MP3|MPEG Audio Layer III}} which has become the de-facto consumer audio format for many.
+| A series of formats using {{w|lossless compression}}. PNG and TIFF are image formats that are suitable for photos, but without (necessarily) resorting to reduced accuracy in order to assist compression. WAV is an audio format that stores the raw audio samples uncompressed, unlike compressed formats like {{w|FLAC}} which can reduce the size of an WAV file by 50% while still being lossless.
-ZIP is a generic compression algorithm (and the name of the format it creates) that can be used to store any other digital files. Anything put within a ZIP file can be exactly decompressed into the original state later on, although any such file already compressed in some way (such as any of the image formats mentioned in this comic, or other ZIPs) are unlikely to recompress significantly more.
+ZIP is a generic compression algorithm (and the name of the format it creates) that can be used to store any other digital files. Anything put within a ZIP file can be exactly decompressed into the original state later on, although any such file already compressed in some way (such as any of the image formats mentioned in this comic, or other ZIPs) are unlikely to compress significantly more. Additionally, {{w|Lossless compression#Limitations|it must also make some data larger}} due to the {{w|Pigeonhole Principle}} and the fact that it compresses losslessly.
 |-
 | Raw data + parity bits for error detection
 | clone of your cat
-| In the number 135, the sum of its digits is 9. So the number 135 could be written as "1359", for example, slightly increasing the amount of data that needs to be sent. But with the slight advantage that, if the number was tampered with, the parity bits may be able tell you that an error has occured. (Possibly that the parity itself was the digit that was miswritten.) But a change from "1359" to "1539" could not be detected, in this method, when extracting the parity digit and using this to presume that the first three digits are indeed 'correct'.
+| In any data transmission, there is the possibility for errors in the data received. Naturally, there are methods to detect a (limited) number of errors. A simple example are {{w|parity bits}}. Say we wanted to transmit the number 181. In the binary representation of 181 (10110101), there are an odd number of '1's. A simple algorithm might append a bit so that the number of ones in the transmitted message is even, so '101101011' would be transmitted instead of '10110101'. At the receiver, the number of ones in the received message can be summed to determine if an error occurred in the message. If an odd number of ones were received, say '101101010', then the receiver knows that an error occurred because the transmitted signal should never contain an odd number of ones due to the parity bit that was added. Such a method can detect single bit errors, but cannot detect 2-bit errors, because the parity of the message would change twice, swapping back to what it was before.
-There are more reliable means to detect errors, such as CRC-32 (now considered obsolete), MD5 and the much more modern {{w|Secure Hash Algorithm|SHA}}. Such values were alluded to in the Hash Table section. But here they are sent ''alongside'' the data, slightly increasing the amount of data transmitted/stored (in order to establish its accuracy), rather than instead of it and vastly decreasing the amount of 'necessary' data (but leaving the virtually impossible task of performing a correct reconstruction).
+There are more reliable means to detect errors, such as {{w|Cyclic redundancy check|CRC-32}}, {{w|MD5}} and the much more modern {{w|Secure Hash Algorithm|SHA}}. Such values were alluded to in the Hash Table section. But here they are sent ''alongside'' the data, slightly increasing the amount of data transmitted/stored (in order to establish its accuracy), rather than instead of it and vastly decreasing the amount of 'necessary' data (but leaving the virtually impossible task of performing a correct reconstruction).
+However it is done, if the check indicates a problem then you can only request a new copy of the data, hoping that the problems encountered in the transmission can be resolved.
 |-
 | Raw data + parity bits for error ''correction''
 | your actual cat
-| With extra error-checking, there are ways to immediately restore the original data with the given additional data. One method is to 'overlap' multiple error-detection parities such that any small enough corruption of data (including of parity bits themselves) can be reconstructed to the correct original value by cross-comparison between all parity bits and the supposed data. One of the first modern methods developed was {{w|Hamming(7,4)}}, invented around 1950, which was a balanced approach designed to handle the typical error conditions typically encountered at the time and has inspired even contemporary electronic methods of maintaining data integrity. Another practical application of error correction bits would be that present in {{w|QR_code#Error_correction|QR Codes}}, using {{w|Reed–Solomon error correction|Reed–Solomon error correction}}.
+| With {{w|Error correction code|error correction codes}}, there are ways to immediately restore the original data with the given additional data in the event of noise in the transmission medium. One method is to 'overlap' multiple error-detection parities such that any small enough corruption of data (including of parity bits themselves) can be reconstructed to the correct original value by cross-comparison between all parity bits and the supposed data. One of the first modern methods developed was {{w|Hamming(7,4)}}, invented around 1950, which was a balanced approach designed to handle the typical error conditions typically encountered at the time and has inspired even contemporary electronic methods of maintaining data integrity. Another practical application of error correction bits would be that present in {{w|QR_code#Error_correction|QR Codes}} and {{w|Cross-interleaved_Reed–Solomon_coding|CDs}}, both using {{w|Reed–Solomon error correction|Reed–Solomon error correction}}.
 |-
 | Better data
 | my better cat
-| This gives up on the data in question and suggests swapping it for different data entirely. It is no longer about the quality of the transfer of data, but judging the actual data instead. Philosophically, it could be saying that the data or cat are better in some nebulous way, or that it simply is more accurate to what the data is trying to record and represent, in the title text's case saying that Randall's cat more closely represents the essence of "catness."
+| All of the previous examples dealt with preserving the quality of the recorded data. However, {{w|Garbage in, garbage out|they can do nothing if the recorded data are bad}}. If the data are of bad quality, then adding parity bits will ensure that the bad quality data is what is recovered. The only way to increase the quality of the extracted data is to start 'better' data.
 |}

@@ Line 42: / Line 42: @@
 GIF compression is not 'lossy' in the same way, i.e. whatever it is asked to encode can be faithfully decoded, but Randall may consider its limitations (it can only write images of 256 unique hues, albeit that these can come from anywhere across the whole 65,536 "True color" range, plus transparency) to be a form of loss, as conversion from a more sophisticated format (e.g. PNG, below) could lose many of the subtle shades of the original and produce an inferior image. For this reason, GIF format becomes one best left to render diagrams and other computer-generated imagery with swathes of identical pixels and mostly sharp edges (and to utilize the optional transparent mask), for which JPEG compression will create prominant image artefacts. Alternatively, he may just have included it as a joke/nerd-snipe.
 |-
-| {{w|PNG}}, {{w|ZIP (file format)|ZIP}}, {{w|TIFF}}, {{w|WAV}}
+| {{w|PNG}}, {{w|ZIP (file format)|ZIP}}, {{w|TIFF}}, {{w|WAV}}, raw data
 | photo of your cat
 | A series of formats using lossless compression. PNG and TIFF are image formats that are suitable for photos, but without (necessarily) resorting to reduced accuracy in order to assist compression. WAV is an audio format that also does not arbitrarily sacrifice 'unnecessary' details, unlike the more recently developed {{w|MP3|MPEG Audio Layer III}} which has become the de-facto consumer audio format for many.
 ZIP is a generic compression algorithm (and the name of the format it creates) that can be used to store any other digital files. Anything put within a ZIP file can be exactly decompressed into the original state later on, although any such file already compressed in some way (such as any of the image formats mentioned in this comic, or other ZIPs) are unlikely to recompress significantly more.
 |-
-| Parity bits for error detection
+| Raw data + parity bits for error detection
 | clone of your cat
 | In the number 135, the sum of its digits is 9. So the number 135 could be written as "1359", for example, slightly increasing the amount of data that needs to be sent. But with the slight advantage that, if the number was tampered with, the parity bits may be able tell you that an error has occured. (Possibly that the parity itself was the digit that was miswritten.) But a change from "1359" to "1539" could not be detected, in this method, when extracting the parity digit and using this to presume that the first three digits are indeed 'correct'.
@@ Line 54: / Line 54: @@
 However it is done, if the check indicates a problem then you can only seek a new copy (of the data, and/or the parity or hash), hoping that the problems encountered can be resolved.
 |-
-| Parity bits for error correction
+| Raw data + parity bits for error ''correction''
 | your actual cat
 | With extra error-checking, there are ways to immediately restore the original data with the given additional data. One method is to 'overlap' multiple error-detection parities such that any small enough corruption of data (including of parity bits themselves) can be reconstructed to the correct original value by cross-comparison between all parity bits and the supposed data. One of the first modern methods developed was {{w|Hamming(7,4)}}, invented around 1950, which was a balanced approach designed to handle the typical error conditions typically encountered at the time and has inspired even contemporary electronic methods of maintaining data integrity. Another practical application of error correction bits would be that present in {{w|QR_code#Error_correction|QR Codes}}, using {{w|Reed–Solomon error correction|Reed–Solomon error correction}}.

@@ Line 32: / Line 32: @@
 | last 4 digits of your cat's chip ID
 | A Bloom filter is a probabilistic data structure that can efficiently say whether an element is ''probably'' part of the dataset, while it can say "element is not in set" with 100% accuracy. If a Bloom filter is used to represent the contents of a book, reference to the Bloom filter could perhaps reconstruct everything, just by guessing, but in a highly inefficient and potentially inaccurate way. A bloom-filter is like a the last four digits of the cat's ID in that while you can know for sure a cat isn't your cat if it's last four digits don't match, you can't know for sure that it is yours if they do.
 | {{w|Hash table}}
 | your cat's full chip ID

@@ Line 31: / Line 31: @@
 | {{w|Bloom filter}}
 | last 4 digits of your cat's chip ID
-| A Bloom filter is a probabilistic data structure that can efficiently say whether an element is ''probably'' part of the dataset, while it can say "element is not in set" with 100% accuracy. If a Bloom filter is used to represent the contents of a book, reference to the Bloom filter could perhaps reconstruct everything, just by guessing, but in a highly inefficient and potentially inaccurate way.
+| A Bloom filter is a probabilistic data structure that can efficiently say whether an element is ''probably'' part of the dataset, while it can say "element is not in set" with 100% accuracy. If a Bloom filter is used to represent the contents of a book, reference to the Bloom filter could perhaps reconstruct everything, just by guessing, but in a highly inefficient and potentially inaccurate way. A bloom-filter is like a the last four digits of the cat's ID in that while you can know for sure a cat isn't your cat if it's last four digits don't match, you can't know for sure that it is yours if they do.
 | {{w|Hash table}}
 | your cat's full chip ID
@@ Line 60: / Line 59: @@
 | Better data
 | my better cat
-| This gives up on the data in question and suggests swapping it for different data entirely. It is no longer about the quality of the transfer of data, but judging the actual data instead.
+| This gives up on the data in question and suggests swapping it for different data entirely. It is no longer about the quality of the transfer of data, but judging the actual data instead. Philosophically, it could be saying that the data or cat are better in some nebulous way, or that it simply is more accurate to what the data is trying to record and represent, in the title text's case saying that Randall's cat more closely represents the essence of "catness."
 |}

← Older revision		Revision as of 17:54, 29 March 2023
Line 10:		Line 10:

	==Explanation==		==Explanation==
−	~~{{incomplete\|Created by a SUPERIOR FELINE. needs review by software engineer to correct mistakes. should link to the comic on taking the fourier transform of a cat. Do NOT delete this tag too soon.}}~~
	<!-- Specifically "No Idea If There's A Character Limit LMAO": please refrain from removing any more Incomplete tags by yourself and so quickly, and please check your Talk page! And please remove this comment once you've read it. :) -->		<!-- Specifically "No Idea If There's A Character Limit LMAO": please refrain from removing any more Incomplete tags by yourself and so quickly, and please check your Talk page! And please remove this comment once you've read it. :) -->

@@ Line 56: / Line 56: @@
 ==Transcript==
-:[A line chart is shown with eight unevenly-spaced ticks each one with a label beneath the line. Above the middle of the line there is a dotted vertical line with a word on either side of this divider. Above the chart there is a big caption with an arrow pointing right beneath it.]
+:[A line chart is shown with eight unevenly-spaced ticks each one with a label beneath the line. Above the middle of the line there is a dotted vertical line with a word on either side of this divider. Above the chart there is a big caption with an arrow beneath it pointing right.]
 :<big>Data Quality</big>
 :Lossy ┊ Lossless