Talk:2054: Data Pipeline

Explain xkcd: It's 'cause you're dumb.
Revision as of 21:43, 5 October 2018 by (talk)
Jump to: navigation, search

tried my hand at transcipts again, hope i did ok. Nintendo Mc (talk) 15:32, 3 October 2018 (UTC)

Oddly prescient, as always. I've just finished writing a fully automated data pipeline that ingests multiple data sources (both manual and automated input), has API support, a frontend, and email dispatch capabilities entirely in Google Sheets. It was about 3x faster to code than doing it right. 16:48, 3 October 2018 (UTC)

That's so awesome ! Would you come back and let us know if it ever collapses because one of the data sources changes slightly? (or alternatively, that it _doesn't_ collapse and cueball needs to get his shit together?) 01:22, 4 October 2018 (UTC)

Just added a line about how this is a logical continuation of the Code Quality series - given it's the same two people, this should be uncontroversial. Is it worth adding a new category for "Code Quality" to group these (and likely subsequent comics) together? Grimreaperwithalawnmower (talk) 17:20, 3 October 2018 (UTC)

Quite controversial, in fact. I actually found that statement quite questionable and that it should probably be removed (only in part because the title isn't grouping it in with them). Related? Certainly. But a full part of the group? THIS Cueball seems like he's far more capable than Code Quality Cueball. THIS Cueball managed to construct a highly useful piece of software that - until the final panel - did the job they needed. The issue here is the Bobby Tables issue, that he neglected to sanitize the input, i.e. to at least write the program in a way that it could handle variety. The program relies heavily on the exact format of the data it's gathering (a format that he has no control over, it's set by the source). Okay, this suggests he's using prewritten code and connecting it together, but getting code pieces together into a cohesive whole is a considerable feat, showing some programming prowess (far better ability that CQ Cueball). NiceGuy1 (talk) 05:05, 5 October 2018 (UTC)

What could we still add to the transcript? I don't think it really needs any more transcripting so maybe we should remove the marker. Kwonunn (talk) 18:50, 3 October 2018 (UTC)

No comment about the "roll over" text (excuse me if I have the name wrong). I think this is a comment about the shear computing power, battery life and superior connectivity of modern mobile phones compared to laptops. RIIW - Ponder it (talk) 19:05, 3 October 2018 (UTC)

IIRC, it's generally called "hover text." (talk) (please sign your comments with ~~~~)
pretty sure it's actually "title text" Halo422 (talk) 01:09, 4 October 2018 (UTC)
It is actually "title text", though Randall calls it "alt-text" & contrary to W3C recommendations, he seems to use the same text for both. ProphetZarquon (talk) 05:13, 4 October 2018 (UTC)
Here on the Wiki people usually go with "Title text", occasionally "Mouse-over text". Which I like, partially because it's clear what it means, even to the casual visitor, and partially because it highlights my issue: I use these sites on a tablet, don't have a mouse, I can't see the text until I come here. :) RIIW has a point, this needs a paragraph about the Title Text. But no, it isn't saying phones are more reliable, it's a joke that neither should be hosting anything, neither is meant to be online 24/7. NiceGuy1 (talk) 05:32, 5 October 2018 (UTC)
Public service announcement: If you can't read the alt-text because you're on a mobile device, you should try using the mobile version of the website: . Its main features include the ability to tap/click the image to make the alt-text show up, and the whole thing is distraction-free. 16:59, 5 October 2018 (UTC)

Re: superior connectivity of mobile phones, see (talk) (please sign your comments with ~~~~)

This is exactly why I assert that anything hosted from a laptop should at least be considered for hosting from a mobile device instead. It's annoying to me that so many developers still consider a mobile device which has more connected uptime than a laptop to be unsuitable for hosting, say, a text-based game server. It's got a faster connection & more idle processing power than the PCs that used to run some of those game servers; I think my tablet could handle running a BBS Door game, for example. ProphetZarquon (talk) 05:14, 4 October 2018 (UTC)
Just to clarify in the comic, it is generally preposterous to host a server process like this from a mobile device. Mobile devices aggressively suspend computing time when processes are in the background, or when the screen is off, and their battery life is much shorter than that of a laptop. It is far more normal to modify the power settings of a laptop to not suspend when closed, which is doable on all laptops running linux, than to run a central data pipeline on a phone, and any meaningful server process has a dedicated server environment. The comic is a joke, like all of them. Cueball has a history of coding for his personal life that he is trying to apply in an environment where more resilience is needed than he is used to. (talk) (please sign your comments with ~~~~)
While all of the comments about connectivity between a laptop and a mobile device may be valid, I think the joke here is that any serious data processing application should not be running on either - it should instead be operated in a fixed-connectivity server-type environment instead. Ianrbibtitlht (talk) 16:37, 4 October 2018 (UTC)
I concur, the point is that nothing mobile should be HOSTING, hosted files should be on a system designed to be never off and never disconnected. Both a laptop and a cell could have their batteries die, or they have to be permanently plugged in, defeating the purpose of them being "mobile". NiceGuy1 (talk) 05:21, 5 October 2018 (UTC)

Had to fix the description, it stated that Cueball reluctantly agreed with Ponytail's statement when he actually did the opposite, but his hesitation suggests she's correct. NiceGuy1 (talk) 05:21, 5 October 2018 (UTC)

The presence of White Hat is a little mysterious here because he doesn't have any lines. What could be going on? 07:41, 5 October 2018 (UTC)

He was just waiting for the next comic to start :) Hawthorn (talk) 15:46, 5 October 2018 (UTC)

Why is there a six paragraph diversion at the end of this explanation? This may be tangentially relevant, but not enough for an explanation that eclipses the size of the rest of the actual comic explanation. Consider removing it or boiling it down to one paragraph on the general topic with a link. I prefer removing it because the comic doesn't make it's own connection to a wider issue. (talk) (please sign your comments with ~~~~)

I'm not the most organized thinker in the world. A lot of the explanations on this site read like the person who wrote the explanation does not actually have experience with the topic of the comic, which is generally written as if it is something that Randall does have experience with. I tried to fix that on this comic, by sharing background from a place of experience, but I wasn't really sure what the most relevent bits were or how to integrate it into the existing work well. The existing explanation read as if Cueball was simply a horrible coder, when in reality these data pipelines are common things among programming hobbyists, and it takes experience to recognize that they are inevitably a house of cards. They're not inherently bad though: liberal input validation can be used to notify a dev when something goes wrong, so that they can fix it fast, but that needs more foresight than Cueball may have if he is running it off his phone. This data pipeline approach is used in live sites still up today. I'm sorry I'm expressing so verbosely; it's being hard for me to be concise. 21:43, 5 October 2018 (UTC)