iOS Client Side Sync

There’s been much ado about writing sync type services for mobile from the server side, but I haven’t seen much about how to implement them on the client side. This is an overview post to see if there’s more interest in how I’ve done things in the past.

Main point: Map your sync data well with Core Data and use NSFetchedResultController’s.

Hit me up on Twitter if a deeper dive into this is something of interest.

—

Most of my iOS experience has been writing enterprise apps used by large companies as opposed to individual consumers. At NewsGator I wrote the first three versions of NewsGator Social Sites for iOS which connected employees to their companies Microsoft Sharepoint portal. The original was written with iOS 2.1. The amount of data that needed to be “synced” far outreaches anything else I’ve ever worked on.

Social Sites had a co-worker concept that was similar to friends or people you follow on Twitter. Those people could post to a timeline with text, pictures and videos, then have other co-workers comment on those posts (also with pictures and video) – all within the company firewall.

There was also the concept of Communities that had their own timeline of posts and comments along with a list of members.

On top of that there were also Sharepoint libraries of documents, photos, task lists, polls or any N number of object types a company decided to add to their Sharepoint portal. All of these could be associated with both individuals as well as Communities.

An incredible amount of data constantly changing.

I’ve had a lot of experience with SQL databases. I love SQL. But when I had to decide between using SQL or Core Data, I chose Core Data specifically for the use of NSFetchedResultController’s.

My approach was to map the data in Core Data as close as possible to how it was represented on the server. I could then create specific NSFetchedResultController’s for each view controller in the app. All of the “sync” would happen in the background and if something that a particular view needed to know about changed, it would be notified via the NSFetchedResultController.

What this bought me was a separation between the networking code, the local data preservation code and the view controller code.

It worked wonderfully.

It’s been over a year since I’ve work on that code, but if there’s interest, I can put something together that’s more in depth which I can share.

 

Numbers as Unique ID’s

Brent Simmons talking about sync for Vesper:

Here’s the thing: unique IDs for notes and users are 64-bit integers in my system.

This actually surprised me a bit. My old co-worker, Brian Reischl on both the NewsGator RSS Sync platform as well as Glassboard, tweeted this reply:

@brentsimmons 2^53 limit is new on me. This is why Glassboard mandated strings – fewer client side bugs to get blamed on server team. 🙂

— Brian Reischl (@brianreischl) April 14, 2014

The reason we mandated strings is because we learned a tough lesson with the NewsGator platform that broke most of our client apps including mine, NewsGator Inbox. I know FeedDemon was also affected but perhaps NetNewsWire was not, so maybe that’s why Brent didn’t feel the effect of this issue like the rest of us did.

The Day Sync Died

One day in the NewsGator office we all started to notice that our web app was no longer working correctly. I looked at NewsGator Inbox and noticed the same thing. We quickly discovered that our internal unique id assigned to each blog post in the system had become too large for the integer type we were using and had in our documentation. We had to mad scramble to patch the database, the web app and the client apps. I don’t remember exactly what the fix was, perhaps Brian does, but the lesson we both learned was to mandate strings for unique ID’s.

Brent had this thought, but dismissed it based on database bloat and inelegance. That’s a silly argument to me. As well as Brent’s other thought of leaving it to a future platform team to figure out when it becomes a problem. You see the problem now, fix it in a way so that its never a problem in the future.

Imagine if the issue does come up in the future and part of the solution is a client side code change. If the problem isn’t noticed way before it happens, your app could be broken for days while waiting for Apple review. Even if you contact Apple and push the bug fix version through, your users are still out of luck for some amount of time.

As someone who has dealt with this situation in a real world scenario, my suggestion would be to use strings as unique ID’s even if its inelegant and causes some minor database bloat. Its a good trade off to not have to worry about the problem in the future.

Update

I should note that we had two other reasons for using strings for unique ID’s.

The first was to obfuscate any REST API calls so that guessing a number wouldn’t somehow get you access to data you weren’t suppose to see. Our security infrastructure rendered this issue moot.

The second was the idea that a client shouldn’t care what’s in a unique ID. The client should just treat the ID as an opaque string. Should the server ever need to encode additional data into an ID, which the NewsGator Platform eventually needed for performance, it shouldn’t effect how clients interact with that ID. I realize Vesper generates some ID’s on the client (which I find to be an intriguing design decision) so this concern doesn’t apply, but I wanted to further explain our approach.

The Night I Rocked Out With Kim Deal

I grew up in Dayton, Ohio.

When I was 17, I worked at a restaurant called Friendly’s. My friend Mike and I scooped ice cream while my other friend Eric was a line cook. We worked with this other line cook named Ben Schelker.

Ben was 10 years older then us and was in a local band called Candyass. His previous band, The Oxymorons, was fairly well known in Dayton so Candyass was being invited onto billings with major up and coming bands. One of the first Candyass shows I went to they opened up for a band named Sebadoh at a strip mall club less then a mile from my parents house.

Dayton at the time was being compared to the next Seattle for its music scene. We had the likes of Brainiac and Real Lulu tearing up the bars around the University of Dayton ghetto. We also had a band named Guided by Voices who were starting to see some real commercial success outside of Ohio.

Ben was friends with Robert Pollard and was asked to join a multi band billing at a jazz club in Dayton.

Benefit for the Winos
June 2nd, 1995
Gilly’s Jazz – Dayton, Ohio
Ticket Price: $8

Eric, Mike and I bought our tickets and headed into the show. We knew Candyass and GBV were on the bill, but there was another band opening named Tammy and the Amps.

As soon as they hit the stage (with a half empty club) this was our conversation:
“Is that Kim Deal?”
“No… Well..”
“That guitar case has Breeders spray painted on it.”
“Yep. That’s Kim Deal.”

Kim and her twin sister Kelly are from Dayton. They had a band called The Breeders. Kim had spent a lot of time in Boston with another band she had founded called The Pixies but both were now back in town. Kelly had just been busted for heroin so the Breeders were no more (at least at that time). In response, Kim started Tammy and the Amps. This show turned out to be their first major gig. They were awesome.

Candyass played after. It was always fun to watch Ben play. He was a damn fine rocker.

The last band was GBV and after Ben’s set he met up with my friends and I. As we were waiting for GBV to start, another friend of his named Kim joined us.

Kim Deal. And she hung out with us for the entire Guided by Voices set.

I remember standing there, a 17 year old kid going into his senior year in high school thinking: “You were in The Pixies. You were on David Letterman. You toured with U2. How am I hanging out with you?”

Infamy
GBV was already known for their drunkenness during shows. This show was over the top. People were jumping on stage and singing the songs themselves. Even Ben got into the action.

At one point, Robert Pollard was laying on his back “signing”.

My friends and I were all sober and in the middle of a drunk rock show meltdown.

A local band beat writer for the Dayton Daily News was at the show. He was an obvious fan but his review, which was printed two days later, tore into the show as the worst and an embarrassment to the band.

Guided by Voices, having a great sense of humor, bootlegged the show on vinyl with the DDN review on the jacket. It’s a very sought after GBV bootleg.

For me, I’ll always remember it as the night I rocked out with Kim Deal.

One of the best nights of my life.

via http://www.gbvdb.com/album.asp?albumid=1005

 

KIS My YAGNI

Rudy Lacovara’s Smart Guy Disease post is a fantastic read. I’ve encountered many of those situations myself, but when you think about it in the context of Scott Hanselman’s Analysis Paralysis: Over-thinking and Knowing Too Much to Just CODE, I think it brings the overall problem more into focus.

I don’t believe that smart, seasoned, experienced programmers create overly complex solutions because they want to appear smart (though admittedly some I’ve met have). I believe Scott’s answer that we over-analyze problems, then over-architect the solution is the real culprit.

When I’m starting on a new piece of code, I often find myself thinking through and solving issues that will most likely never occur. Its a hangover effect from some bad situation I ran into previously that I simply want to avoid again.

For myself, a combination of the solutions from both posts is the best.

YAGNI – Ya Ain’t Gonna Need It

You’re a seasoned, experienced programmer who has seen many pitfalls and issues. That’s awesome. Use that experience to evaluate the actual problem you’re solving now and potential pitfalls that are eminent. Forget the rest. If / when they become problems you can solve them then. You already know how to solve them, but why complicate the problem at hand.

KIS – Keep It Simple

Simple code is easy code. Easy to learn. Easy to maintain. Easy to transfer. If you can’t explain your solution to a programming problem by pointing to well known and accepted coding standards, you most likely over thought the problem and out thought yourself. I love the standard Rudy has:

…your boast should be that your architecture is so well designed that any junior developer can come in off the street, understand the basics in a day or two, and be productive within the week.

That standard is my idea of true programming expertise. Been there. Done that. Made it simple.