Talk:2739: Data Quality - Revision history

172.70.85.224 at 02:02, 7 March 2023

2023-03-07T02:02:21Z

172.70.85.225 at 02:01, 7 March 2023

2023-03-07T02:01:01Z

SDSpivey at 01:39, 7 March 2023

2023-03-07T01:39:14Z

172.70.114.78 at 16:54, 24 February 2023

2023-02-24T16:54:32Z

172.70.114.79 at 16:51, 24 February 2023

2023-02-24T16:51:52Z

172.70.114.78 at 16:48, 24 February 2023

2023-02-24T16:48:41Z

162.158.62.140 at 16:40, 24 February 2023

2023-02-24T16:40:03Z

172.70.85.224 at 19:20, 22 February 2023

2023-02-22T19:20:48Z

172.71.178.65 at 11:23, 22 February 2023

2023-02-22T11:23:41Z

162.158.238.6 at 08:24, 22 February 2023

2023-02-22T08:24:07Z

@@ Line 13: / Line 13: @@
 :GIFs are lossy in the very act of creating them: the actual colors of the real object have to be smashed down into (I think it’s) 256 different colors, resulting in an image that even human perception recognizes as crappy. Even the so-called ‘lossless’ formats such as PNG are lossy in the act of creation, just not as drastically as GIFs. A truly ‘lossless’ format would have to specify the exact intensity of every wavelength of electromagnetic radiation emanating from every atom of the original object. Good luck with that. [[Special:Contributions/172.71.151.99|172.71.151.99]] 01:00, 18 February 2023 (UTC)
 :::GIFs can only have 256 colors per *frame*, but can have many frames, so 16,777,216 (256^3) colors total should be possible. [[User:SDSpivey|SDSpivey]] ([[User talk:SDSpivey|talk]]) 01:39, 7 March 2023 (UTC)
-::::Temporal dithering? Don't know if that's the term for it, but it's thenone I'd use to describe it.
+::::Temporal dithering? Don't know if that's the term for it, but it's the one I'd use to describe it.
 ::::And I remember trying that on a BBC Microcomputer, messing with fast direct video-memory copying and also the interupts to get the high-res but monochome MODE 0 (1-bitplane, but with some choice of foreground and background colours that are used that can be changed fairly rapidly, as well as in horizontal bands) to create a disconcerting effect (I wouldn't subject an epileptic to it!) that could still approximated at least a 3-bit colour-mode. Half the colour-res of  MODE 2, twice that of MODE 1, but vertical dot-res twice that of the latter and ''quadruple'' that of the former. IIRC. [[Special:Contributions/172.70.85.225|172.70.85.225]] 02:01, 7 March 2023 (UTC)
 ::It's subjective whether formats (even .gif) can be recognised as 'crappy'. The display format may further tune down everything so that something defined with 65536 colours is more like 256, or it could work well with any given stippling/halftoning/dithering to produce something more like the better original than the file data strictly allows (even from 6bits-per-pixel, or 3) when viewed at sufficient remove. And a .gif of a block-coloured diagram is notably better than a typical .jpg of one, despite the technically superior palette the later has. (Nobody says that an image has to be from a real-life subject, with all kinds of missing data, such as photons thst happen to hit the gap between CCD pixels but might be considered important and might well have been captured with the Mk 1 Eyeball and significantly 'noticed' by the nerves and ultimately the respective processing usters of the brain behind it... Which has a complete set of 'analogue lossiness' to it, anyway.) [[Special:Contributions/172.71.242.203|172.71.242.203]] 16:37, 18 February 2023 (UTC)

@@ Line 12: / Line 12: @@
 :I think that's the point. [[Special:Contributions/172.68.50.203|172.68.50.203]] 20:12, 17 February 2023 (UTC)
 :GIFs are lossy in the very act of creating them: the actual colors of the real object have to be smashed down into (I think it’s) 256 different colors, resulting in an image that even human perception recognizes as crappy. Even the so-called ‘lossless’ formats such as PNG are lossy in the act of creation, just not as drastically as GIFs. A truly ‘lossless’ format would have to specify the exact intensity of every wavelength of electromagnetic radiation emanating from every atom of the original object. Good luck with that. [[Special:Contributions/172.71.151.99|172.71.151.99]] 01:00, 18 February 2023 (UTC)
-::GIFs can only have 256 colors per *frame*, but can have many frames, so 16,777,216 (256^3) colors total should be possible. [[User:SDSpivey|SDSpivey]] ([[User talk:SDSpivey|talk]]) 01:39, 7 March 2023 (UTC)
+:::GIFs can only have 256 colors per *frame*, but can have many frames, so 16,777,216 (256^3) colors total should be possible. [[User:SDSpivey|SDSpivey]] ([[User talk:SDSpivey|talk]]) 01:39, 7 March 2023 (UTC)
+::::Temporal dithering? Don't know if that's the term for it, but it's thenone I'd use to describe it.
 ::It's subjective whether formats (even .gif) can be recognised as 'crappy'. The display format may further tune down everything so that something defined with 65536 colours is more like 256, or it could work well with any given stippling/halftoning/dithering to produce something more like the better original than the file data strictly allows (even from 6bits-per-pixel, or 3) when viewed at sufficient remove. And a .gif of a block-coloured diagram is notably better than a typical .jpg of one, despite the technically superior palette the later has. (Nobody says that an image has to be from a real-life subject, with all kinds of missing data, such as photons thst happen to hit the gap between CCD pixels but might be considered important and might well have been captured with the Mk 1 Eyeball and significantly 'noticed' by the nerves and ultimately the respective processing usters of the brain behind it... Which has a complete set of 'analogue lossiness' to it, anyway.) [[Special:Contributions/172.71.242.203|172.71.242.203]] 16:37, 18 February 2023 (UTC)
 :I encoded the records you wanted transferred to your department's systems into a standard GIF format file. Would you prefer an MJPEG video? [[Special:Contributions/172.70.114.79|172.70.114.79]] 16:51, 24 February 2023 (UTC) EDIT: You're right, though. Maybe Randall has experience with color loss using GIF. In the 90's GIF was a compressed photography format, smaller than BMP. 16:54, 24 February 2023 (UTC)

@@ Line 9: / Line 9: @@
 GIF's aren't lossy either, though often other formats can't be converted to GIF without discarding information. [[User:Bemasher|Bemasher]] ([[User talk:Bemasher|talk]]) 18:27, 17 February 2023 (UTC)
 :I think that's the point. [[Special:Contributions/172.68.50.203|172.68.50.203]] 20:12, 17 February 2023 (UTC)
 :GIFs are lossy in the very act of creating them: the actual colors of the real object have to be smashed down into (I think it’s) 256 different colors, resulting in an image that even human perception recognizes as crappy. Even the so-called ‘lossless’ formats such as PNG are lossy in the act of creation, just not as drastically as GIFs. A truly ‘lossless’ format would have to specify the exact intensity of every wavelength of electromagnetic radiation emanating from every atom of the original object. Good luck with that. [[Special:Contributions/172.71.151.99|172.71.151.99]] 01:00, 18 February 2023 (UTC)
 ::It's subjective whether formats (even .gif) can be recognised as 'crappy'. The display format may further tune down everything so that something defined with 65536 colours is more like 256, or it could work well with any given stippling/halftoning/dithering to produce something more like the better original than the file data strictly allows (even from 6bits-per-pixel, or 3) when viewed at sufficient remove. And a .gif of a block-coloured diagram is notably better than a typical .jpg of one, despite the technically superior palette the later has. (Nobody says that an image has to be from a real-life subject, with all kinds of missing data, such as photons thst happen to hit the gap between CCD pixels but might be considered important and might well have been captured with the Mk 1 Eyeball and significantly 'noticed' by the nerves and ultimately the respective processing usters of the brain behind it... Which has a complete set of 'analogue lossiness' to it, anyway.) [[Special:Contributions/172.71.242.203|172.71.242.203]] 16:37, 18 February 2023 (UTC)
 :I encoded the records you wanted transferred to your department's systems into a standard GIF format file. Would you prefer an MJPEG video? [[Special:Contributions/172.70.114.79|172.70.114.79]] 16:51, 24 February 2023 (UTC) EDIT: You're right, though. Maybe Randall has experience with color loss using GIF. In the 90's GIF was a compressed photography format, smaller than BMP. 16:54, 24 February 2023 (UTC)
 Someone needs to add a table describing all the formats in the chart. [[User:Barmar|Barmar]] ([[User talk:Barmar|talk]]) 19:29, 17 February 2023 (UTC)
 :Yep. It needs a description of each point on the graph. I'm on my phone though... and feeling lazy after shoveling snow.

@@ Line 12: / Line 12: @@
 :GIFs are lossy in the very act of creating them: the actual colors of the real object have to be smashed down into (I think it’s) 256 different colors, resulting in an image that even human perception recognizes as crappy. Even the so-called ‘lossless’ formats such as PNG are lossy in the act of creation, just not as drastically as GIFs. A truly ‘lossless’ format would have to specify the exact intensity of every wavelength of electromagnetic radiation emanating from every atom of the original object. Good luck with that. [[Special:Contributions/172.71.151.99|172.71.151.99]] 01:00, 18 February 2023 (UTC)
 ::It's subjective whether formats (even .gif) can be recognised as 'crappy'. The display format may further tune down everything so that something defined with 65536 colours is more like 256, or it could work well with any given stippling/halftoning/dithering to produce something more like the better original than the file data strictly allows (even from 6bits-per-pixel, or 3) when viewed at sufficient remove. And a .gif of a block-coloured diagram is notably better than a typical .jpg of one, despite the technically superior palette the later has. (Nobody says that an image has to be from a real-life subject, with all kinds of missing data, such as photons thst happen to hit the gap between CCD pixels but might be considered important and might well have been captured with the Mk 1 Eyeball and significantly 'noticed' by the nerves and ultimately the respective processing usters of the brain behind it... Which has a complete set of 'analogue lossiness' to it, anyway.) [[Special:Contributions/172.71.242.203|172.71.242.203]] 16:37, 18 February 2023 (UTC)
-:I encoded the records you wanted transferred to your department's systems into a standard GIF format file. Would you prefer an MJPEG video? [[Special:Contributions/172.70.114.79|172.70.114.79]] 16:51, 24 February 2023 (UTC)
+:I encoded the records you wanted transferred to your department's systems into a standard GIF format file. Would you prefer an MJPEG video? [[Special:Contributions/172.70.114.79|172.70.114.79]] 16:51, 24 February 2023 (UTC) EDIT: You're right, though. Maybe Randall has experience with color loss using GIF. In the 90's GIF was a compressed photography format, smaller than BMP. 16:54, 24 February 2023 (UTC)
 Someone needs to add a table describing all the formats in the chart. [[User:Barmar|Barmar]] ([[User talk:Barmar|talk]]) 19:29, 17 February 2023 (UTC)
 :Yep. It needs a description of each point on the graph. I'm on my phone though... and feeling lazy after shoveling snow.

@@ Line 12: / Line 12: @@
 :GIFs are lossy in the very act of creating them: the actual colors of the real object have to be smashed down into (I think it’s) 256 different colors, resulting in an image that even human perception recognizes as crappy. Even the so-called ‘lossless’ formats such as PNG are lossy in the act of creation, just not as drastically as GIFs. A truly ‘lossless’ format would have to specify the exact intensity of every wavelength of electromagnetic radiation emanating from every atom of the original object. Good luck with that. [[Special:Contributions/172.71.151.99|172.71.151.99]] 01:00, 18 February 2023 (UTC)
 ::It's subjective whether formats (even .gif) can be recognised as 'crappy'. The display format may further tune down everything so that something defined with 65536 colours is more like 256, or it could work well with any given stippling/halftoning/dithering to produce something more like the better original than the file data strictly allows (even from 6bits-per-pixel, or 3) when viewed at sufficient remove. And a .gif of a block-coloured diagram is notably better than a typical .jpg of one, despite the technically superior palette the later has. (Nobody says that an image has to be from a real-life subject, with all kinds of missing data, such as photons thst happen to hit the gap between CCD pixels but might be considered important and might well have been captured with the Mk 1 Eyeball and significantly 'noticed' by the nerves and ultimately the respective processing usters of the brain behind it... Which has a complete set of 'analogue lossiness' to it, anyway.) [[Special:Contributions/172.71.242.203|172.71.242.203]] 16:37, 18 February 2023 (UTC)
 Someone needs to add a table describing all the formats in the chart. [[User:Barmar|Barmar]] ([[User talk:Barmar|talk]]) 19:29, 17 February 2023 (UTC)

@@ Line 6: / Line 6: @@
 : Maybe it was to be about '''cuckoo filters''', which are probabilistic data structure alternative to classic Bloom filter, which are based on space-efficient variants of cuckoo hashing? --[[User:JakubNarebski|JakubNarebski]] ([[User talk:JakubNarebski|talk]]) 14:05, 20 February 2023 (UTC)
 ::Hash tables don't have to store the original data at all, technically; they are commonly done as hash table->KEY:DATA or hash table->KEY:Pointer to data (or suchlike), but hash table->present is a valid hashing scheme, which results in a likely verification that you have the right data (but not guarunteed because collisions) but no way of reconstructing the data itself. [[User:Mneme|Mneme]] ([[User talk:Mneme|talk]]) 02:25, 21 February 2023 (UTC)
-:He’s casually referring to the hash conflict situation in common implementations of hash tables: the table of hashes, not the whole structure. You have O(n) lookup speed proportional to the impact of uniqueness lost in the hash lookup. The point is that this is the same way that bloom filters {which also usually need a source of truth to be useful) are used. The two concepts perform the same function but with different degrees of lossiness, different widenesses of matching. [[Special:Contributions/162.158.62.140|162.158.62.140]] 16:40, 24 February 2023 (UTC)
+:He’s casually referring to the hash conflict situation in common implementations of hash tables: the table of hashes, not the whole structure. You have O(n) lookup speed proportional to the impact of uniqueness lost in the hash lookup. The point is that this is the same way that bloom filters {which also usually need a source of truth to be useful) are used. The two concepts perform the same function but with different degrees of lossiness, different widenesses of matching. [[Special:Contributions/162.158.62.140|162.158.62.140]] 16:40, 24 February 2023 (UTC) EDIT: it also leaves it ambiguous that it could mean a table of hash functions outputs as you suggest, where hashes have often been thought of as uniquely identifying data that is not recoverable (this does require a sufficiently constrained situation but is often used), where bloom filterd are thought of as ambiguously referring to multiple items. I can imagine it being more clear to leave out the word table. [[Special:Contributions/172.70.114.78|172.70.114.78]] 16:48, 24 February 2023 (UTC)
 GIF's aren't lossy either, though often other formats can't be converted to GIF without discarding information. [[User:Bemasher|Bemasher]] ([[User talk:Bemasher|talk]]) 18:27, 17 February 2023 (UTC)

@@ Line 38: / Line 38: @@
 :''Very'' relevent to the parity ones. (Leads me to believe it's a scale of "amount of provided data to represent original data". You send less than you really ought to, the more left you go, you send more than you should ''technically'' need to as you go to the right. Checksums add a little bit extra, once you get to them, and ''correcting'' checksums (hamming bits, etc) are significantly extra overhead. The whole 'better data' is basically "send a similar amount of newer information, or even more, on top of the original".) [[Special:Contributions/162.158.34.71|162.158.34.71]] 12:55, 20 February 2023 (UTC)
 :: But that's about adding information to the file (which happens to be mitigation against the potential future loss of data) - not directly about the loss of data itself. [[Special:Contributions/172.71.178.65|172.71.178.65]] 11:23, 22 February 2023 (UTC)
 Since when is CRC-32 obsolete? [[Special:Contributions/162.158.238.6|162.158.238.6]] 08:24, 22 February 2023 (UTC)

← Older revision		Revision as of 08:24, 22 February 2023
Line 37:		Line 37:
	The opening sentence of the explanation, about data loss in transit, seems a bit irrelevant to the comic, which is only concerned with lossiness in information due to format. [[Special:Contributions/172.70.91.197\|172.70.91.197]] 10:40, 20 February 2023 (UTC)		The opening sentence of the explanation, about data loss in transit, seems a bit irrelevant to the comic, which is only concerned with lossiness in information due to format. [[Special:Contributions/172.70.91.197\|172.70.91.197]] 10:40, 20 February 2023 (UTC)
	:''Very'' relevent to the parity ones. (Leads me to believe it's a scale of "amount of provided data to represent original data". You send less than you really ought to, the more left you go, you send more than you should ''technically'' need to as you go to the right. Checksums add a little bit extra, once you get to them, and ''correcting'' checksums (hamming bits, etc) are significantly extra overhead. The whole 'better data' is basically "send a similar amount of newer information, or even more, on top of the original".) [[Special:Contributions/162.158.34.71\|162.158.34.71]] 12:55, 20 February 2023 (UTC)		:''Very'' relevent to the parity ones. (Leads me to believe it's a scale of "amount of provided data to represent original data". You send less than you really ought to, the more left you go, you send more than you should ''technically'' need to as you go to the right. Checksums add a little bit extra, once you get to them, and ''correcting'' checksums (hamming bits, etc) are significantly extra overhead. The whole 'better data' is basically "send a similar amount of newer information, or even more, on top of the original".) [[Special:Contributions/162.158.34.71\|162.158.34.71]] 12:55, 20 February 2023 (UTC)
		+
		+	Since when is CRC-32 obsolete? [[Special:Contributions/162.158.238.6\|162.158.238.6]] 08:24, 22 February 2023 (UTC)