Category: Uncategorized

The Web and Internet Security

Part 1 of n

Hi Everyone!  I’m back!  The last couple years have been a hell of my own making, through the depths of depression and back again.  I’m still a bit burnt out, but definitely on the upswing.  But enough about me.

Pointedly ignoring the clusterfuck that is US and world politics right now, I’ve decided to speak on a topic that is near and dear to everyone’s heart, privacy and security.  This is something I know a bit about, having worked with private data in web and mobile, as well as a few hobby attempts at secure chat and connection apps.  Rather than go deep into the tech, in which I’m far from an expert, I’ll talk about the concepts and issues that I’ve seen.

As a disclaimer, I work for Facebook.  In this article I’ll touch on them, but I am not in any way shape or form speaking for them.  These are strictly my personal opinions.

So let’s start with a model of your basic web or mobile app.  There’s a server somewhere, or more likely a server farm (the “cloud”).  There’s the Internet itself, a web (see what I did there?) of interconnected special-purpose devices whose job it is to shuttle data around, based on attached metadata.  Then there are clients, which can be mobile devices, tablets, desktops, or a web page or custom app.  Not surprisingly, this is called a “client-server” architecture.

Whether the client is a web page or a mobile app, the people who funded the system want to see how their investment is doing, how they can make it better, and bring the site back up quickly if something goes wrong.

The servers will keep logs of how many requests were made, what information was returned, and in case of a problem, what failed and where.  The clients will also send back information, about how people are using the app, and what information was requested.

Together, these analytics let management know who to advertise to, what their interests are, and how best to serve the users and the investors.

You’ll note I’m being careful not to put a value judgment on any of this.  The bottom line is, content costs money to produce, serve, and maintain, and if the people fronting that cost aren’t making any money on that investment, they’ll find another investment.  If they know what people want from their application, it’s in their interest to provide it, since that will bring more users.

Which brings us to ads.  Once you know what people like, you have a good idea of what kinds of things they’ll buy.  If you can target the advertising you vend, they’re more likely to click your ad links, buy that stuff, and thus make you more money.  That also means that the space you reserve for advertisers will be worth more, since the clickthrough rate is higher.  In advertiser lingo, putting an ad in front of someone’s eyes is an impression, and buying the product is a conversion.

So what can go wrong?

Let’s start with the advertising itself.  Since impressions and clickthroughs are worth money, advertisers will go to great lengths to maximize their number.  This means annoyingly distracting ads, popovers, popunders, and other obnoxiousness to ensure that ads are seen and maybe even accidentally clicked.  Fortunately content providers are starting to become aware how much badly behaved ads affects their goodwill, and are taking steps.

As far as “data mining” and such, the legitimate use of my data for marketing purposes is the least of my concerns.  If I don’t want to buy something, I simply won’t, no matter how well targeted the ad.  If tracking my data means the content will be more interesting to me, all the better.

There are concerns that some sites are in fact customizing or suppressing content in order to make users feel better.  I can’t speak for my employer, but once again I’m not too worried.  I’ve seen plenty of dissenting content on my feeds, and whenever I see news that seems relevant to me I’ll check the source before believing or forwarding it.  Checking sources will be the topic of a future article, but not too far in the future.

Then there are hackers.  Some sites use woefully inadequate security, or don’t keep up with the latest exploits as they are found, and so leave themselves open.  Once someone gets in, everything is potentially up for grabs.  Also, hacking has become big business, so there are a lot of people, in and out of governments, who are doing it.

Finally, the government themselves.  No matter who is in office, there’s going to be some abstract noun that’s used as the latest witch hunt.  This means that every byte of data, once it’s left your computer (and maybe even before that, soon-to-be-upcoming article), can be copied and analyzed at any stop along its way.

A subpoena can also be issued to retrieve the analytics data that companies have on you.  Without a warrant, and without notifying you.

The primary takeaway here is if you don’t want something to be public, don’t post it.  Even if you set the most restrictive privacy settings, and the site you’re posting it to assures you that your private data remains private and yours.  You’re just one hacker, one subpoena, and/or one “Our Terms of Service Have Changed” email away from your deepest secrets becoming front-page news.


An excerpt from my upcoming book

There are problems that are complex.  These are the ones that have many moving parts, and there is an emergent behavior that seems greater than the sum of its parts.  Complex problems are fun to solve, and when you’ve done it, you have a great sense of accomplishment.  For example, the Mandelbrot Set is based on a relatively simple equation, and is literally infinitely complex.

Complexity is an intrinsic property of a problem.  Looking at the problem itself, what is the minimal solution for the entirety of the problem?  As simple as possible, but no simpler is the mantra.  As long as the problem is fully stated, the complexity is known, and it can be completed.

Complication, on the other hand, is the set of external considerations, the extrinsic properties.  I could add an Apache server to a linux box and throw it on the internet with a 20-line PHP script to serve content (thus satisfying the full complexity of the problem), but there’s a 99% chance the linux box will be pwned within 24 hours.  Security, therefore, is a complication.  Pre-existing code that you don’t understand and have to work with is a complication.  Development process is a complication.

Complications comprise the ‘work’ part of work.  Some of them you’ll know about, like adding enough security so that your web site is hard to hack, or the code review process you need to get your work into master.  Many of them you won’t know about – a new OpenSSL vulnerability is discovered, an edge case in the code you inherited, a sudden change in the specification.

I will state that at best you can know 40% of the complications involved in a task.  That means that at least 60% of your work is unknown at the time you commit your code to master.  You need to anticipate that 60% and allocate resources to handling these complications.

It seems like the Religious War Du Jour is the discovery that Object-Oriented Programming Sucks, and anyone who wants to write code that works should use a Functional paradigm.

The blogs and websites are full of semi-contrived examples where OOP has gone horribly wrong.  And indeed, the problem they are trying to solve is not a good one for OOP, and the result is hacky spaghetti code.  Then they show the same problem solved in a functional language in 10 lines.

The lesson here isn’t that OOP sucks and FPL is a panacea.  The lesson here is:

Use the right tool for the job.

If I hired a contractor to do work on my house, and she insisted that everything can be done with a pair of pliers, I’d be skeptical.  Yes, you can grab a screw head and turn it with pliers, but a screwdriver will be a lot easier and safer.  If I were paying her by the hour, I’d only be paying for one hour.

In the many years I’ve been coding professionally, I’ve found that both good and bad architectures are self-sustaining.  A bad architecture will force you to pass data all over to get it where you need to, copy-paste nearly identical code in multiple places, and implement hacks to get what you need done.  The learning curve will take forever, and you’ll feel dirty.  Your self-esteem will fall since you’re writing buggy code you’re not proud of, and you don’t feel confident it will work for all edge cases.

A good architecture will have clear interfaces, separation of concerns, comments and unit tests.  When you make a mistake it’s almost immediately underlined in red by your IDE because the static checker can catch the common problems.  When you enter ‘git push origin master’, you do it confidently because the result feels right.

Is a circle a subclass of ellipse or vice versa?  Depends on what you’re doing with them, but if you have a renderAsPDF() method in either of them, you’re probably doing it wrong.

Wow, it’s been three years since my last post!!  Also, a lifetime.

I spent about half the time working at Twitter, the other half at AOL and Facebook.  I moved from NYC back to our house in the Bay Area, CA.

Last year I was privileged1 to participate in a layoff, which was presented to me as a termination for lack of performance, along with, unbeknownst to me until later, half the office.  At first I took it really hard, but then I decided to see what I could learn from the experience.

First of all, I realized that my performance was just fine – in fact, while I was being told how bad I was doing, I was also dealing with a severity 0 failure, starting with finding the root cause, contacting the appropriate developers, and keeping management aware of progress.  During one of these management calls, I had forgotten to ask some specific question (I don’t remember; it doesn’t matter) and got screamed at by the manager for two minutes about how sloppy my work was.  Meantime the sev0 was fixed over a weekend, and a hotfix saved a lot of people trouble.

But the main learning here is that as a senior engineer and beyond, the primary skillset isn’t how well you program, it’s how well you navigate the work social graph.  At the end of Thinking Physics, Lewis Carrol Epstein has an appendix on the growth of business organizations that was really enlightening.  A small startup will have one manager, the CEO, and say 3-8 engineers.  Everyone will be producing and working hard, and the CEO will have the additional task of securing funding and marketing the product.

Eventually, if lucky, the CEO will need to hire an office manager, an accountant, and more engineers.  At this point, she or he will probably have to hire an additional manager or two.  Epstein estimates it to one manager per 10 employees, whose job it is to coordinate resources, and goals for those employees, and communication with other managers and the CEO.

If the startup makes it to mid-level, around 100 employees, an amazing thing happens.  We now have 10 managers, which means we need to hire a middle manager, whose sole job is to coordinate the 10 managers, and is removed from the ‘front lines’ by a full level.

This is used as an argument that smaller companies can move faster and work more efficiently, by minimizing the number of communication lines that have to be maintained.  But the reality is, companies grow.  Sometimes they’ll split off smaller companies, but often they just get bigger, and hence further removed from their original mission, toward maintaining their own size and maximizing profit and shareholder value.

What this means is that you will likely work in large companies, and have to take on a good portion of this communication yourself.  Senior and Staff engineers have to coordinate with engineers in other groups, because the managers may not have the bandwidth to fully immerse themselves in the technical details, all they can do is provide the introductions and step back.

If you want to accomplish your cross-team goals, it’s critical therefore that the engineers you need to work with want to work with you.  Going to social events, taking people out for drinks, remembering their kids’ names, now are part of your way of life.  You want them to hear your voice, or see your Instant Message, and think happy thoughts.  Otherwise they’ll put you off and delay you, and your project will fall behind, and you get “(barely) meets expectations” at your next review instead of “exceeds.”  Even if you’re a topnotch programmer!

And this is what happened to me way back when – I failed to understand the importance of the social graph.  I would let my moods and my paranoia influence my interactions with peers in other offices to the point where I couldn’t get my code reviewed and kept missing deadlines.  Which of course led to my mood getting even worse.

But in truth, a failure is only a failure if you don’t learn from it.  Learn from my failure, and don’t fail yourself!
1 That’s sarcasm, just in case you couldn’t tell

Happy 2012!

Let’s hope the last year of the Mayan calendar is better than the previous!  And that someone is busy carving another 35,000 year calendar.

Like this Blog?

The more people my Dashboard shows me are reading this, the more time I will spend crafting interesting articles, and the more likely I am to buy the “no ad” option!

My plan is to blog about things I’ve learned in my 20+-odd year career as a software engineer, as well as my excursions into the Arduino, embedded microcontrollers, and electronics.

Tell your friends, or tell me!

thanks, Rob D

Apologies for the delay, there’s a lot of backstory I’m preparing for this – The ABI is much more complex than I originally thought, and I would rather be complete and late, than inscrutably on-time.

Welcome and Introduction

Welcome to Rob’s Adam’s Apple, where I will blog about my trials and travails as a programmer and occasional hardware hacker.

I am currently an iThing developer at Barnes and Noble, working on the Nook Reader application. I am also developing an Arduino clone called the Carduino, and am building a Nixie Tube Calculator.

Having over a quarter century of programming experience, there are a few things that I have found to be very important to be a successful programmer.  The first is communication.  This one still trips me up.  When not talking to computers in their respective native tongues, you are talking to:

  1. Other Programmers
  2. People who do not necessarily understand what you are doing, but want to control how you do it.  Also known as Managers.
  3. People who likely don’t understand what you are doing, but want to change how you do it.  Also known as Customers, Clients, and/or Stakeholders.
  4. Other people.  These may include friends, significant others, potential managers, or potential significant others.

I will concern myself with the first 3, the last is up to you.

When communicating with other programmers, be respectful.  You may not understand why they didn’t initialize a variable, which caused it to have a random value and crash the system during a customer demo.  Which may make you think they are stupid or incompetent.  Even if they are, it won’t help to accuse them of it.  When talking to anyone, always consider how you would feel if they said what you’re about to say to you, in the tone of voice you’re about to use.  Now think of how you’d rather hear it and say it that way.  These people may have your back some day – do you want them to tell you about your uninitialized variable first, or the CEO?

When talking to managers, honesty is the key.  These people need to represent you up the food chain.  To do this, they need to know what’s going on.  Resist the temptation to say that the project that you feel will take four weeks can be done in 1.  It can’t, and you may be fired after week two when it’s still only half done.  Better to find out what they need implemented next week, then plan out the following three.  They’ll be less upset, and in the long run you’ll get the reputation of dependability, which leads to promotions.

Stakeholders don’t care how you’re implementing your web fetches.  If you tell them you’re performing asynchronous background fetches, they won’t understand and will feel frustrated.  You always want stakeholders to associate you with warm fuzzies.  Delegate the bad news and compromise to the manager, and come through for the clients.

I definitely speak autobiographically here – hopefully some of you reading this can get the benefit of my mistakes 🙂