Twenty eight years later, do we still remember? January 28, 2014 at 13:50

 

Twenty eight years ago today, the Space Shuttle Challenger made its final flight.

As young as I was when it happened, it still had a profound impact on me. At that time, my family tended to watched every launch, and I can still remember where I was in the living room about the time the stack came apart on the television screen, not quite sure what had happened until I saw the tears in my mother’s eyes.

It still leaves me saddened when I think about it at length.

Then the sadness will pass, and I will immediately become angry. Angry because, while the road to the stars is hard and sometimes will be costly, we need not make it any harder than it has to be.  The Challenger crew need not have died, it was a completely preventable incident.

It was -8 F when I woke up this morning here in Michigan, which is fitting on today of all days, because it was a similar record freeze at Cape Kennedy that caused the O-ring failure, which lead to the SRB leaking rocket exhaust at the SRB support struts, the failure of those struts, which caused the Space Transportation System Flight 51L’s stack to break apart.  This was a failure that was predicted and documented.  That engineers had brought to their management their concerns and were  ignored for political expediency, time and time again.  Management forgot their lessons.  They forgot accountability, standing up for what’s right, making the truth known.  That you’re supposed to listen to the people working for you.  That losing face is always better than losing lives.

They’d been given these lessons before, you see, when the Apollo 1 fire claimed Grissom, White, and Chaffee.  They knew better due to that sacrifice, but decided to go out and win one for the Gipper anyway.  As a result, Scobee, Smith, Onizuka, Resnik, McNair, Jarvis, and McAuliffe were all lost to that fire in the sky.

Today, on this coldest of Southeastern Michigan days, we should remember these things in our own lives, that accountability, the truth, and doing what’s right are what we should live by.  To show that the Challenger crew’s deaths were not in vain, for they paid the coldest, hardest cash possible to teach us lessons that in many ways  we really already knew, but often chose to ignore. That the truth (especially in physics!) does not care, and will play out no matter what spin we place on things.

If we remember this, and live this, then we will not discount that price, and their memory will still be honored.

“For a successful technology, reality must take precedence over public relations, for nature cannot be fooled.” -Richard Feynman

 

Irons in the Fire (Projects and upcoming geekery) April 13, 2012 at 12:49

It’s been a grind of a week this past week, so I haven’t had a lot of spare braincells to finish polishing a few post drafts that I’ve got sitting in a holding pattern right now, so it’s been a bit radio silent.

I don’t want to lose posting momentum, it’s what killed me last time, so here’s what I’ve been working on:

Ensemble RX/TX SDR - This is a software defined radio kit project for the amateur radio HF bands that I’ve been working on over the past couple of months.  I’ve got a big post about it coming up soon that I hope to have video and screencaps shot of the unit sometime this weekend.  I’m going to be rewinding coils and hopefully getting the transmit portion of this up and running this weekend.

Following closely on the heels of that, I’ve jumped on the bandwagon and ordered one of the Realtek-chipset based USB DVB-T tuner widgets from an eBay seller that should be arriving any day now.  Apparently the things can be used as 64-1700 MHz software-defined radio receiver radio front-ends.  All I have to say to that is…HELL YES.  So I ordered one.  The prices have gotten jacked up from the $10-15 they were originally selling for, but for $28, if it works half as well as they say it does, it’s still a steal for experimentation purposes.

A Cobweb antenna for the house – Living in a narrow urban lot, with power and telecom lines running liberally through the backyard (I have to contend with my power lines as well as my neighbors), putting up a decent HF antenna can be problematic.  Given that ham radio operators in the UK have to deal with gardens that are in similar situations (or smaller!), I went hunting for solutions that UK hams have come up with, and ran across this antenna design.  I have all the electrical parts required, I still need to pull the trigger on ordering the fiberglass from Max-Gain (My original source doesn’t stock what I want any longer).

Near future projects:

Building a Pennywhistle HF amplifier kit from TAPR.  Giving the Ensemble a little more transmit oomph, 1W probably won’t end up cutting it.

A switchable low-pass filter for the amplifier kit and the Ensemble RX/TX – I’m probably going to go with something like the LPF-100 R3A  from HF Projects, just so I don’t have to source all the parts and gin up a PCB, I don’t have a ton of time and my layout skills kinda suck.

A Z-Match QRP antenna tuner for tuning up antennas to the radios – This one may get supplanted by finding a good deal on a QRP tuner at a hamfest.

Linkage! April 6, 2012 at 17:51

I added a small pile of links to the site, they’re all blogroll, utilities I use on a regular basis and can recommend, or games.

Sometimes I find the most interesting stuff in the datacenter. April 5, 2012 at 17:29

 

I had to escort a telecom tech into our first floor datacenter at WeSupportHospitals the other day.  I don’t actually visit that room all that much anymore, it’s pretty full, and ongoing renovations mean that the configurations are pretty static for the moment.  Looking around for a chair, I ran across this:

 

(Apologies for the crappy cellphone cam pictures.)

 

It’s a 36,000 RPM flywheel energy storage system.  That’s a high vacuum port on the side so there’s no air to slow down the flywheel while spinning.  The rotor is suspended on on magnetic bearings, with nothing touching it.

It seems like something that should be mounted in a particle accelerator chamber somewhere.

This one is actually only the defective one.  As I understand it they went to fire it up the other day, and during the startup process, the maglev bearing controller failed to power up the magnet, but still somehow reported all-good, so the actuators released the rotor to float and spin up, and promptly dropped the rotor, rendering it useless.  The replacement was overnighted. :)

Just one of those things that scream futuristic, I guess.

 

 

 

Gluing the Internet together #1, DNS Glue records April 1, 2012 at 09:45

(This post was inspired by a conference call from tonight, wherein the author had to explain how the following worked, during an overnight major change window.)

I’ve been meaning to do a bunch of simpler articles about basic topics that most networking folks and admin folks should know, but don’t seem to much of the time.

I often joke that a lot of my job is gluing the Internet together, and how I describe what I do for a living to folks that have their eyes glaze over when I say the phrase “network engineer”.

But that’s not the sort of gluing I’m talking about today.  Instead, I’m going to talk about something that if we didn’t have, the Internet wouldn’t work the way it does today.

It’s DNS glue records, and understanding it is essential for any DNS administrator who has to spend any time working somewhere where they are totally self-hosted.  The thing is, it’s amazing how many DNS administrators don’t understand how this works.

DNS, for those that don’t know, is a hierarchical system.  Starting all the way at the top are the root servers.  These are what are in your root.hints file.  Those are the servers that your DNS server goes to query first to go find out who the heck deals with the next domain down the list (com/net/org, country ccTLD registries, etc).  Then, the buck passing continues until the dns server either arrives at the correct server hosting this information, or gets to the last server in the domains, and gets a NXDOMAIN response back.  Basic DNS 101 type stuff, right?

This assumes, however, that everything is well known, that is, SOMEONE in the chain has the IP/IPv6 addresses for the next set of servers down the line.  As an end-user hosting a domain with some given web provider, you’ll be using their name servers, so you don’t really have to worry about that being complete, you’re just the person on the end, and as long as their nameservers are findable, what do you care, right?

Now assume you’re working somewhere, such as an ASP or cloud provider, and you ARE that hosting company.  Chances are, you are also hosting your own DNS, maybe you’ve got your offsite DNS providers or secondary site, like a good hosting provider should, but otherwise you’re totally self-reliant for DNS, nobody is hosting your records.  So how the heck does anyone know how to reach your domain?

This is where DNS glue comes in.

Let’s run an example to show how this is supposed to work:

1. A DNS server goes looking for mail.example.com.

2. It has no idea where to find information about example.com, or even the com domain (unlikely, but let’s say the server just booted).

3. It does have a root.hints file.  Aha!  It can go ask those guys running DNS root!  So it picks one of those IP/IPv6 addresses listed there for the root and asks for where the .com domain is.  It gets back a list of nameservers to go ask for the information, along with the IP/IPv6 addresses of those nameservers.

The bold text is the part you should be paying attention to.  It’s  the first DNS glue records handed out.  Root has to know enough and pass on enough info about .com for the DNS server asking for it to go and even talk to com.

4.  Our intrepid DNS server goes and asks the .com servers, “I’m looking for who handles example.com, can you hook me up?”  Like an air traffic controller at a busy ARTCC handing a flight off to a local airport for landing, it responds with “example.com’s nameservers are ns1.example.com, ns2.example.com, contact at 127.0.0.5 and 127.0.0.6 respectively, good day!”

Bold text again!  This is the second glue record that’s being passed to the requesting DNS server.

5.  DNS server asks ns1 or ns2.example.com for mail.example.com, gets back a response, responds to the client that asked it for the info with the IP address for that hostname.  It’s also cached the info for .example.com and .com, knowing that it’ll just go directly to those sources if it needs information the next time around, speeding the process up.

Now, most of the time, step 4 is where everything goes horribly wrong for a given domain.  If the glue for .com were screwed up, there’d be a big chunk of the Internet coming to a screeching halt, and there are major steps taken to prevent this from ever happening.

Step 4′s setup is the one handled by the registrant of the domain, if they’re self-hosting their own DNS, instead of making it someone else’s problem.  They not only have to tell the .com registry what the nameservers are, they also have to feed that little bit of sticky bootstrappy information, the DNS glue record.   Thankfully, most registrars will validate that any given nameserver is resolvable or has its own glue in place before accepting it as a valid nameserver.

Where this goes horribly wrong is when a company moves servers around in their IP addressing space, or changes ISPs and doesn’t have their own address assignment.  All too often, because DNS servers don’t tend to change too much (stability is a good thing here, and DNS doesn’t need much in the way of resources to run for most companies), the institutional memory has been lost of what setting up or changing a self-hosted domain requires.  Any other domains can just use the example.com nameservers, so we’ll just specify those!  The DNS glue, sitting it its dusty virtual jar way back on a top shelf, has been forgotten.

So they update their A records for those servers, and figure that their work is done.  It isn’t.  After everybody’s DNS caches expire, they’re going to go looking for who the heck is authoritative for example.com again. They’ll get their handy helping of glue from .com, which hands out the old addresses, and this is where the whole process falls flat on its face.  The DNS lookup times out, the browser put up “page cannot be displayed” messages, and an angry customer wants to know why the heck their stuff stopped working.

The things that kept a major change from possibly blowing up on us tonight are a combination of the institutional memory having a hazy recollection, specifically, one of the apps guys asking “hey, didn’t we have to tell the registrar something the last time we changed some nameserver addresses?”), just before I asked on the conference call “You did update your glue records, right?  Tracing via dig isn’t showing that it’s not showing up with the new info, and it’s been a little bit (glue changes seem to propagate quite quickly for com/net/org).”  The mention of the words “glue records” got the attention of the admin handling the changes, after some explanation, they understood what the issue was, and sure enough, they hadn’t done it because they didn’t know they needed to.  Five minutes went by after the correction was made, and the new addresses were being properly sent out by the .com servers.

It’s not that I have dumb co-workers, in fact, I work with some really awesome folks, it’s been the best job I’ve had in the ten-plus years I’ve had in the IT industry.  It’s just that infrequent changes don’t get remembered well by many people, and sometimes things go wrong.  I’ve just been bit by this issue enough times that it’s the first and last thing I check when moving DNS stuff around.  So what’s the fix to help keep such things from happening?  Documentation, documentation, documentation.  When the domain is set up in a self-hosted configuration, note what it took to get that set up, and put that on the internal wiki, a three ring binder,a PDF, SOMETHING.  You do have a documentation share that all your folks use, right?  Put it there, in your procedures directory (you have one of those too, yes?).

The moral of this story is, if you make radical changes to how you host you DNS, always check your DNS glue and document how that got set up in the first place.   Your customers will appreciate you for it.

 

W5YI, SK March 30, 2012 at 15:35

 

I just found out that Fred Maia, W5YI passed away this past week at 76, of cancer.

While I don’t expect that the few of you who are reading this nascent website yet know who this is, a lot of us in the amateur radio community do.  He was a major mover and shaker, writing a newsletter called “The W5YI Report” for over 20 years from 1978 until 2003, and was instrumental in organizing one of the two largest license exam coordinators for the amateur radio community in the United States.

I’m pretty sure I took my exams from a Volunteer Examiner team affiliated with the W5YI VEC, and I know a lot of other hams have as well.

So to Fred, I say, 73′s OM.  May the rare DX be good, the QRM and the QRN be minimal, and may all your message traffic reach its final recipient, wherever you are.  You got a lot of us started, and while many of us never met you, we may not have gotten licensed if you hadn’t done what you’d done.  You probably have done more to keep this hobby and service alive than most of us ever will.

The ARRL news release covers this a lot better than I possibly can here.

 

 

 

 

 

1GB SFP in SFP+ slots on Cisco Nexus 5000 series switches at 10:00

 

One very common thing that I keep getting asked is “Why doesn’t this 1 gigabit SFP module work in this 5548 series Nexus switch? The specs say it will!”

It’s actually something that’s really really simple to fix.

Symptom:

1Gbit SFP in SFP+ slot, “show interface brief” displays the following for the port in question:

Eth1/11       1      eth  access down    SFP validation failed       10G(D) –

Cause:

The reason this is happening is for one of two reasons:

1.  You’re on an older 5020, and you’re plugging into a port numbered 17 or higher.  In which case, find a lower numbered port to use and see step 2.  If you have no other ports from 1-16, you need to find another switch to plug into. :)  Older Nexus 5000 series can only support SFP in SFP+ in the first 16 slots.  Don’t ask me why, it’s just how they’re built.  This is fixed in the newer switches, including the 5548.

2. You’re on any given 5000 series switch, and the port doesn’t know it’s supposed to be a 1Gbit port.  The ports can tell they have an module plugged into them, and they’re looking for certain validation bits from the module to say if it’s a SFP or an SFP+ module.  They can’t, however, autodetect what sort is actually there, all they know is that it’s not passing its validation check.

Solution:

The fix for this is pretty easy.  All you have to do is tell the switch what the speed of the SFP it’s got plugged in is running at.

Configuration:

#conf t
Enter configuration commands, one per line. End with CNTL/Z.
# int Ethernet1/11
# speed 1000

#show interfaces brief

Eth1/11       1      eth  access down    Link not connected         1000(D) –

The other side isn’t up yet, but at least the switch can see that the SFP is valid and good.  Proceed with configuring the switch normally.

NX-OS is something that I’ve been spending a  bunch of time working with lately, there will be more configuration tidbits and tips/tricks coming down the pipe in the future.

Like spaceflight? Like cartoon explosions? Do I have a game for you! March 28, 2012 at 12:00

I don’t have a heck of a lot of time or spare cash for serious hardcore gaming lately(Even though someone decided to buy me a Minecraft license recently).  Keeping the Counterstrike twitch reflexed honed is something I just can’t do, so I’ve ended up drifting into a lot of indie strategy, puzzle and the occasional action game.  Many of them are rather inexpensive, but one shouldn’t parse that as “cheap”.  The Humble Indie Bundle has been getting a lot of my gaming cash, the value and concept can’t be beat.

One that’s not part of those deals, but has gotten a lot of my time and has provided a TON of entertainment is Kerbal Space Program.  The game’s concept is rather simple.  Take some rocket parts, most of them dodgy, and build a spacecraft capable of getting the little green Kerbals into orbit, and then eventually to the Moon.  If you fail, there’s explosions and parts flying everywhere, sometimes to hilarious ends.

Contrast this with the hardcore spaceflight simulator Orbiter.  A game that’s so focused on the realism of spaceflight that sound is a 3rd party add-on because the original developers never thought sound support was important. A game with a learning curve so steep, going out in the backyard and building your own Saturn V looks attractive.  I do love it, but it’s not for the casual gamer, it is Serious Spaceflight Simulation, and it really isn’t intended as a game, per se.

Enter Kerbal Space Program (KSP).  It’s really structured for casual gameplay, but the sandbox nature of the spacecraft builds leads to some folks trying to accomplish some rather serious goals (geostationary “satellite” placement, for example).  It uses a somewhat Newtonian model of physics, so running the math about orbital mechanics on paper (or in your head, if you’re so inclined) works.  There’s a built in orbital display showing your craft, it’s path and orbit, and other objects in space that are still orbiting for those not so mathematically inclined.  Control is largely manual and seat-of-the-pants, though there’s autopilot and auto-launch mechanisms available via the plugin system (more on this in a second).

The game is currently under active development, 0.14 being the latest release, and the first release that requires that you pay for it in order to get it.  Prior releases are still free, the developer has committed that this will be the case going into the future.  This being said, 0.14 was a major release for KSP, and adds one very major, long-awaited feature to the game, persistence.  Whereas prior releases had you have your spaceflight from start to end in one session, with only one spacecraft in orbit at any given time, 0.14 features a persistent world, one where orbiting craft (and space junk!) stay aloft between exits.  This also means that you can have multiple launches and crews now.  There’s something fairly awesome about landing one craft on the Mun (the in-game name for the lunar body orbiting the planet), and then launching and landing a second craft near the first.

There’s another new feature, plugins, also major, but it’s still really in an Alpha state, so I’m not counting it as release material yet.  But!  Even with barely any documentation, the community has written some really awesome plugin modules for the game!  Missing autopilots, orbital rendezvous computers, automatic staging, it’s been really impressive the functionality the users of the game have written within a week of the plugin API being announced.  I highly recommend the MechJeb plugin, if for nothing else than the expanded instrumentation it gives you.

This brings me to the community aspect of this game.  It’s had a very active community presence since Day 1, probably due to the discovery of the game by the Something Awful Games forum folks.  Community-produced spacecraft parts have been plentiful and well received.  Because it’s so early in the development, sometimes the learning curve gets a little steep for some of the more advanced aspects, a read-through of the site’s message boards is highly recommended.    The boards are quite active and quite helpful with getting into the game.

Future game support will include an in-game funding model, where you get increased funding for reaching certain goals successfully, and every part having a cost when you build them.  $15 is currently what the game costs at the store, and for the moment, entitles you to free upgrades for life.  I’ve certainly gotten my money’s worth in a fairly short time, and if playing with a cute simpler spaceflight game is your thing, Kerbal Space Program is the ticket.

DO YOU HAVE A BLOG? March 27, 2012 at 14:34

“Do you have a blog?”

This question, which has now become a legendary inside joke with a few friends and I, started a few years ago at Notacon.  A friend of mine and I were enganged in our usual network security Gedankenexperiments when we were approached by a gentleman claiming to be a professor of computer science.

And he brought to us one of those ideas that everyone seems to drool over, but nobody seems to implement well (save Ricochet), which is mesh networking.  This gentlemen wanted to do this with lasers (which scales poorly to begin with) to get around the evil government and their nefarious “Internet Kill Switch”.  One does tend to encounter these sorts of folks at hacker-cons, so it’s not exactly a new experience to have someone want to get all grassroots on their bandwidth, often in wildly impractical ways.

Going from mildly irritated to somewhat amused by his Laser Mesh Internets to save the world idea, we started dissecting it, looking for holes.  And holes we found, plenty of them.

Toward the end of the conversation, he kept asking us over and over “DO YOU HAVE A BLOG?”  The look on his face when we both finally told him “no”, was that of disappointment.  We’d also gathered a small audience at that point as we debated the topic, which, at the time, I also found strange (I now know better).

At dinner that night, I found out that he’d been chatting with various hackers at the con, asking the same question.  “DO YOU HAVE A BLOG”, as a result, went down in history.

I’d actually given it some thought, trying to write a professional blog/website/whatever, but figured that tech blogs were a dime-a-dozen, so shelved the idea at the time.

Fast forward the timeline to recently, and a few things have recently happened.

1.  I started reading the blog of Joe The Peacock (another Notaconner and a Fark web designer).  The guy really resonates with me, and his “Only you’re going to be the reason you succeed” style message has really been hitting home.

2.  I’ve started getting asked why I don’t write, from a lot of folks.  Apparently there’s enough folks really entertained by my tech and personal anecdotes, that they want an online anthology, or something.

3.  Documentation.  The last, but certainly nowhere near the least reason.  I get a lot of questions, via email, IM, IRC, G+, etc.  A lot of them tend to be the same questions, over and over again.  This is going to be a place for writeups, useful tech tidbits that I commonly use, strange issues that I’ve encountered, things like that.

That’s not all it’s hopefully going to be, though.  I’ve got some ham radio projects in the pipeline, and I’ve got other things that I’ll probably talk about.  My goal is to make at least one post per week at first, then trying to update twice weekly.

With all of those reasons pushing at me, I decided to give it a go.

 

“DO YOU HAVE A BLOG?”  Yes, now I do.