Totally awesome software for iPhone and OS X

Monday, January 19, 2009

I wish I had a screencap of this...

I've got Bloomberg buzzing away in the corner for a bit of company (freelancing can be lonley) and they just flashed this across the screen.

Bloomberg INSIGHT : Global trade slowdown bad for growth.


Is that really so very insightful? Do traders still pay a few thousand a month for terminals to recieve helpful updates like that? Perhaps I should ditch this coding lark and consider a career in financial journalism.

Tuesday, January 6, 2009

Watch out where the huskies go...

This is interesting.

The iPhone Dev Team are largely responsible for the genius software that jailbreaks and unlocks iPhones. I use it because I live in a fairly remote region of France with intermittent mobile coverage and had a UK iPhone. So, rather than paying international mobile rates or waste time and money signing up for an Orange (France Telecom) deal it was just easier and cheaper to get a pay as you go sim and run pwnagetool (the software that removes the carrier restriction on the phone).

Works perfectly.

Anyway, a few people have taken the freely available work of the Team and packaged it up as an 'unlocking solution' and are selling it. Selling it for £50 no less! This, despite the iPhone Dev Team not accepting donations and having a license clearly stating that the code is freely available but should NOT be used for commercial purposes.

It's an interesting case because I presume the offender (I won't give them the google karma of linking to their site) is assuming that as it's a legally grey area they won't have any recourse. However, I'd argue that the Dev Team have a pretty good case and here's a couple of reasons why - I'm not a lawyer though, just an interested geek...

1. As far as I know unlocking phones is not illegal. As long as you fulfil your contract with the carrier, the phone is not stolen and you legally connect to a new network I don't think there's a problem. Unlocking phones for profit may be an issue, but that's not the fault of the software and is more likely to count against the person selling the software, not those giving it away.

2. The Dev Team can prove that they wrote the software and released it under a non-commercial license and they can also prove that the people that stole it have hardly made any attempt to obfuscate the fact. So, they can't say, for example, that they looked at the source and wrote their own version and even if they could they'd still be on dodgy ground. In fact, all they seem to have done is hacked a few properties strings around and changed the odd copyright message.

3. If the Dev Team or users of the software have somehow violated any of Apples terms and conditions that's a separate issue and would be the case if you used the £50 (still can't get over that figure) copy of the same software anyway. So it's irrelevant to the case.

The Dev Team are looking for legal advice :

Also if there are any UK based lawyers or student lawyers that could help us with this, then please contact us at blog@iphone-dev.org of course student lawyers can’t give us specific legal advice, but even your informal opinion helps.


So if anyone who reads this can help, I'm sure you know what to do. Finally, I'd like to thank the Dev Team for spending their time and money hacking on this and I'd like to spend a little time frowning in the general direction of people who rip off their (and other open source) work for profit.

Tuesday, December 16, 2008

La Philosophie pour les Nul(l)s...

As the dark, cold nights are drawing in I've been throwing logs on the fire, sitting in a comfortable chair with a glass of Cardhu and thinking a little about null values work (or should work) in Java. For example, if you define a method as returning a String, conceptually speaking, what should it do? Should it return only String instances, or should it return either an instance of a String or null.

I've long been used to the way Java happily passes null around and have developed a hell of a lot of code over the years that checks for null, as I'm sure have most people. But, wouldn't it make more sense if the implicit contract of the return type specified by the method were adhered to?

Recently, in some cases, we've been writing what I guess you could call 'null-safe' objects (all fields pre-initialised and setters that don't allow values to be nullified). This on the face of it seems conceptually valid, the methods return what they say they'll return and there's a lot less null checking to do when using them. Objects written like this are ideal for e.g. sticking into freemarker templates as then the template designers don't need to care about checking whether or not values exist because they always do.

Problem solved?

Not really. There is an obvious conceptual difference between zero and nothing for a Long value, or between 'no string at all' and an empty string. And if you're going to 'protect' your objects like this, how do you signal that you've done so to other users of your API?

I think most people are comfortable with a certain synonymity between a null string and an empty string, but it gets trickier for other types (and let's not even get into primitives - which can't ever be null in Java). The problem seems to be that Java has no proper way of handling null other than throwing an unchecked exception when you accidentally encounter it.

Microsoft's venerable .NET has the concept of nullable types which (at least for value types, structs and enums) allows you to specify that a type can, infact, be null e.g. int? j = null; But this, with the exception of allowing nullable value types is little more than a syntatcially nice way of doing null checking (or, indeed the 'null-safe' object approach I identified above).

Is there a better way?

Well, it appears that other languages may have a more cromulent approach. Haskell, Scala etc. avoid the use of null (well, technically Scala does have null, but generally it's not used).

Scala prefers instead the use of a simple 'Option' monad that has two sub-classes 'Some' and 'None'. Option is type-safe, so you can have an Option of any type (but only of the type you define it to be). 'None' is essentially a type-safe null (a singleton object, like Nil in Ruby) and 'Some' is a holder for the actual value. Combined with Scala's filtering and pattern matching, we can explicitly exclude 'None' values from operations or act differently (insert a defult value instead, perhaps) depending on how we want to proceed. I'm no Scala expert, but this does seem a saner approach.

Another similar approach is taken by Objective-C, in which Nil is an object. The nice thing about the way Objective-C handles Nil (and objects in general) is that you can happily send messages to Nil and nothing bad will happen. Unexpected things may happen, but nothing bad :)

Oops *oops = nil;
NSLog(@"Oops is doing %@", [oops doSomething]);

This will probably, from memory, print out something like "Oops is doing (null)", not brilliant. But surely better than a runtime exception?

So, in short, I don't really know what the answer is here... In some cases you need to know about null, nil and empty values and in other cases you don't, that's a given. And you can, kind of, achieve this duality in Java but it's annoying that null itself is a special case. It's not an object, it isn't a proper type and calling methods on a null object throws a runtime exception. You could, I suppose, almost say that this approach was similar to Scala's, in that every java Object is implicity an 'Option', 'Some' is the value (if any) and null is synonymous with 'None' (except it's not type-safe etc. etc. etc.) and you do all the pattern matching and checking yourself and if you don't, or get it wrong the whole thing blows up in your face.

I can't help thinking there could have been a cleaner way to handle all of this.

Monday, December 15, 2008

Fancy learning books.

Bill the lizard has a great post listing books that programmers don't really read . In the post he points out that lots of people claim to have read titles such as the GOF patterns book, but haven't really :) He also suggests a list of books which programmers actually (probably) have read, such as Code Complete.

I'll confess to owning a copy of The Art of Computer Programming and never having read it. I have worked through some of the maths with a paper and pencil and I've used it as an infrequent reference but that's about it. I'm saving the rest of it for my retirement...

As for his other suggestions, I agree wholeheartedly. If you're looking for a comprehensive list of worthwhile holiday reading you could do worse than filling in the gaps in your book collection with these suggestions.

On another note, I noticed the following comment from 'anonymous'.

I graduated from Imperial(UK) around 2 years ago and have been programming for a well-known company in a well-paid job for just over a year now. I have read none of these books. Hurray!


Sadly, this attitude is quite prevalent amongst 'career coder drones' in the various industries I've worked in. It seems bizzare to me that people would be working in a field that they have little or no interest in - other than that of getting paid at the end of the month.

I suppose you can get away with it, but, as they say, 'those who don't learn from history are doomed to repeat it' and I'm sure it's unmotivated, bookshy coders like this that are responsible for the 'well-paid' contracts in 'well-known' companies that I keep getting offered to help fix their broken wheel reimplementations...

Wednesday, November 26, 2008

The Closet JRubyists

Brian Tatnall writes an interesting article on JRuby. In short, he says it's fantastic. It looks like they used it to solve memory and performance issues in a financial processing app. Interesting read, makes me want to play a bit more with JRuby.

Friday, November 21, 2008

Is not using an ORM premature optimisation?

Corey Trager writes in this blog post about how he's "stockpiling ammunition" (collecting articles) to use in a workplace argument (discussion) against using an ORM solution in a project.

Now, apart from what he describes in his post, I don't know much about his project and in his case he may well be right to argue that using an ORM, for them, is a complete waste of time and effort. I don't know.

But in the general case (i.e. almost everywhere) I personally think that chosing not to use ORM is a premature optimisation. Simply giving hand waving arguments about performance without metrics and claiming that using ORM is just following the herd is not going to sway my point of view on this and here's why...

1. Performance

Ok, so you could argue that, even without supplying metrics, not using an ORM can ultimately be more performant for some scenarios (which may or may not represent real life usage patterns). But that's like arguing that writing your app in assembly language is going to run faster than the same thing done in Java, Ruby, Python etc. It may be technically true, but you're going to burn up a lot of time and build in a lot of complexity right from the start. So in terms of developer performance and code maintainability you're going to lose out.

'Whaa?', I hear you say, 'who cares if it's easy for the developers to work with, I want blazing fast systems not lazy, bigass developers futzing around with the 'tool du jour' just because it makes their lives easier'.

Yes, but also remeber that you don't necessarily know at this point what the performance characteristics of your application need to be. Let's say you're building a web based API as Corey is. Do you know upfront what the average load will be, how it will be used and so on. Sure, you can guess or even do a little research and get some approximate figures and patterns. Hell, you could even run some benchmarks that test the relative performance of direct connections to a database against an ORM, but they won't tell you anything meaningful at this point. You may even discover that the direct connections are marginally quicker, but it's not a real life usage scenario and half of the (free) benefits of using an ORM probably won't even be brought into play by your test scenarios.

One thing I've discovered in my ten or so years of building big ass apps is that the bottleneck will at some point be the database. Yep, the database itself. Whether it be locked up doing some big query or update (hey, ORMs make locking problems easier to solve too), sending data down the wire to your app servers, out of memory, disk space, available connections or whatever. It's very rarely the app that you'll find yourself waiting on. Of course, there's a case that some of these problems may be the fault of the your app but this is going to be true regardless of whether you're talking to it via some kind of ORM library or a bunch of connections and queries you're managing yourself.

Anyway, we'll come back to performance so let's move on :

2. Software Developers != DBAs

In my experience developers don't think or act like DBAs. They don't often write optimal SQL and they tend to think in terms of code when trying to solve problems in SQL. The opposite is also true, DBA's often write funny code, I once worked with one who was given the task of parsing text files and, of course, set about solving it using an RDBMS. The parsing worked, but took hours and it was quickly replaced with a few lines of C or Perl (I forget which) that did the same job in seconds.

So, abstracting developers from the database can be a good idea, ORMs generally don't make too bad a job of optimising queries and laying out schemas in most cases and they at least follow consistent principles in how they go about such things. The same can't usually be said of people. And besides, ORM frameworks keep getting better as they evolve, so you get the optimisation benefits (collected over years of other peoples' real life scenarios) for FREE every time you update the library. You can't easily build that in yourself and you almost certainly can't build it for free.

By all means, design the database properly, or better still get a DBA in to do it for you. There's no rule anywhere that says you have to model your entity objects in some hyper generic, polymorphic uber-pattern that creates thousands of tables with many-to-many relationships between them and slows to a crawl trying to inflate an object.

You can be pragmatic here. If you have an existing schema or a nice DBA designed one, why not just map your entities directly to that? Think a bit about how your entity model is going to look when mapped out as a database schema, change it if necessary. Most ORMs offer some kind of migration feature that will create and update your database schema for you.

If you know nothing at all about databases, then letting the ORM create you one based on your object model isn't going to be any worse (and probably better) than you'd manage without.

'Wait a minute!' you say, 'That's not much of an abstraction.' Well, no it's not and yes it is, with an ORM the real abstractions are the mechanisms for acquiring database connections, querying relationships, fetching and updating data, managing transactions, caching things and more. The responsibility of knowing that your design is stupid and won't work when translated into a database schema ultimately rests with you.

Again, this is also true if you decide to go down the 'roll your own' route, you just don't get any help along the way.

3. Caching

Big apps are all about caching. Caching everywhere. Facebook have something like 25 terabytes of in memory caches and you're probably going to want some too at some point. Most, if not all, ORM implementations have caching built in, they cache queries, entities, relationships and all the things that you're going to need to remember to find and optimise and cache yourself if you don't use one. They deal with stale objects in the cache for you and take the load of development, testing and debugging all this crap from off your shoulders. Of course, you can tweak them for performance later (when you know what needs doing) and add more caching as you need to other parts of your app, but they're all there ready to go.

Caching is quite a hard problem, but if you decide on rolling your own anyway you'll find caching libraries everywhere and they're not that difficult to work with, but applying them consistently and appropriately is.

This is done for you when using a decent ORM.

4. When you're not using an ORM aren't you really just using an ORM?

If you're not using an ORM to manage the 'interface' between your app and your database. What are you doing? You're probably getting a connection, querying data (being careful to only query for the data you need and doulbly optimising the query), doing some stuff with caches and putting it in a sort of code based representation of the data, cleaning up any used resources and sending it on it to the user. The representation, probably a DTO of some sort, is doubtless an object.

So you're bascially just writing your own specialised ORM then?

Except you've written (and need to maintain) more of your own code to do it. It used to be the case that ORMs required a tightly coupled set of confusing mapping files in xml or similar and involved a lot of work to do apparently simple schema updates. But this hasn't been the case for some time. The two ORMs I use most frequenty JPA/EJB3 (Hibernate) and ActiveObjects are mapped using annotations within the code. But instead, if you don't use an ORM, you've now got a codebase that's tightly coupled to your database. Of course, there may be some benefits to this, but I doubt they'll have much real value. If you've made your database abstract layer more generic than this, then, well why didn't you just use an off the shelf solution?

'Yeah, but what if you're doing data intensive stuff, that's where ORMs really suck. Like getting tens of thousands of rows and performing calculations on them?' You ask.

Well any ORM worth it's salt will let you use...

5. Native queries

Sometimes the whole coding thing just breaks down and SQL is more appropriate, as it usually is for large set based operations. Imagine a scenario where you have to do something to huge datasets and need to update them all at once. You could (but probably don't want to) use an ORM to do something like.


//get every MyEntity in the database
//increment someting by 5
for(MyEntity entity : entityManger.find(MyEntity.class)) {
entity.setSomething(entity.getSomething()+5);
entity.save();
}


This will be slower than the equivalent UPDATE statement on any given RDBMS, if for no other reason than the database doesn't have to transfer the data between itself and your app. This should be pretty obvious.

The arguments for not approaching this specific type of problem this way are the same irrespective of whether or not you're using an ORM. And no ORM worth using is going to tie you in to iterating a massive result set just because it's somehow more 'OO' to do so.

Again, ORMs allow you to bypass their funkiness and call directly through to the underlying database. Breaking the abstraction? Not really, just being pragmatic. Maybe I'm wrong, but as mentioned earlier I don't think the purpose of an ORM is to allow you to create 'whatever crazy OO model you dream of' and magically and transparently map it to a database with little thought or effort and everything will perform wonderfully. The abstraction is the abstraction of not having to deal with the whole kit and kaboodle of connection pools, caches, query optimisations etc. except when you have to.

It can scarcely be denied that the supreme goal of all theory is to make the irreducible basic elements as simple and as few as possible without having to surrender the adequate representation of a single datum of experience. - Albert Einstein


6. What else?

Other complaints that I've seen levelled at ORMs include the following. I'll deal with them quickly as this is becoming a long(er than intended) post...

Concurrent updates : From the 'ammunition' collected in the post that sparked this article, I found this quote, from Bruce Eckel I believe, "Most O/R mappings try to give the illusion that there is just that one Customer object with custID 100, and it literally is that customer. If you get the customer and set a field on it, then you have now changed that customer. That constrasts with: you have changed this copy of the customer, but not that copy. And if two people update the customer on two copies of the object, whoever updates first, or maybe last, wins."

This is simply not true. Hibernate et al. support optimistic concurrency control as much as the next man. How else were you going to solve this? Make your domain objects singletons, lock the row when you 'check out' some data to read it just incase you decide to modify it?

The SELECT * 'problem' : It's a common belief that looking up objects in an ORM involves eagerly loading everything, even stuff you don't want. Again, not true. Completely configurable. Make sensible choices. If you roll your own you're going to need to do this anyway.

7. So, I built an ORM based solution and it totally sucks...

Well, ok, if you're absolutley sure you did your best and the whole thing is falling apart and it completely sucks and it's all the fault *exclusively* of the ORM and you've proven this mathematically and no ammount of tuning and tweaking makes it work.

Then rip it all out.

You should have a clean enough code base, after all you'll be left with POJOs and stuff which you can reuse. I'd hope that your code will be much smaller and easier to refactor as a result of having used an ORM than it would be otherwise. Did I mention that ORMs generally make unit testing and the like much easier as well? Improving code quality overall.

To summarise then:

1. ORMs are not magic, they don't understand intent. They just help abstract away some of the pain of dealing with databases and all the associated cruft that comes with that. You don't have to make random design decisions, if you're stuck with an existing database consider accomodating it a bit in your OO design rather than performing a bunch of crazy mappings. Better still migrate it. Do whatever you'd do if you weren't using an ORM, they generally don't enforce a strict set of rules on how you should use them.

2. Work done by other people, built upon work done by better people and tested in thousands of real life scenarios is going to be better and more bug-free than something you knock up yourself. Every time.

3. Don't immediately wave your hands in the air and claim 'performance sucks'. I've worked on sites using JPA/EJB3 that deal daily with many hundreds of hits per second and don't suffer from performance issues. I've also used ORM in environments where very high processing throughput and heavy calculations were required, again without issue. It may well have been the case that EJB2s container managed persistence was a bit 'merdique' (shitty) but god, that was years ago things have come a long way since those days...

4. If you don't use an ORM and your app isn't running entirely in the database (oracle forms?) then you're going to be building some form of database abstraction layer anyway, whatever you call it or think of it as it's just a custom, one off ORM. That's what it is. I'm sure that this approach could, in the end, be slightly more tuned to your specific needs. But I'd argue that it's a lot of extra work for not much extra gain.

Richy

The Typing of the Dead

Surprisingly, I found myself agreeing for once with Jeff Atwood in his recent blog post regarding typing.

Its one of those things, like ergonomics that is fundamental to what us professional nerds do. Away from the glare of the monitor I also have some other hobbies inluding building and restoring old sports cars and fixing broken analogue synthesizers (wierd mix, I guess). Anyway, you wouldn't approach either of these tasks without some basic understanding of how go about them and some decent enough tools to work with.

Yet people do this every day when they sit down to work in their offices on sub-standard (or even broken) chairs, with crappy office supplied keyboards and mouses and the monitor positioned such that they have to spend the day craning their neck to read whatever it is that people in offices read all day.

So, I thought I'd compile a list of things I find essential in my office.

1. Ability to type - as Jeff points out, as a geek you'll be doing a lot of this. Play Typing of the Dead or Mavis Beacon, or whatever it takes to get you to at least 70-80 wpm _comfortably_.

2. Decent Keyboard. For Macs, I still recommend the KeyOvation Goldtouch keyboard. I've been itching to try a dvorak layout for a while, but then it'd be back to point 1 for me, will definitely be trying it at some point though.

3. Decent Chair. I have a Herman Miller Aeron chair and I love it. When you think about how much time you're going to spend sitting on your fat arse typing you may as well splash out a few quid on a decent, ergonomic chair. They're not always expensive either, I got mine from a company going bust (these may be quite easy to find these days :) ). There are other 'decent chairs' around as well, so make sure you try it out first and pick one that suits you and you think you can spend 8+ hours a day sitting in. Chosing a crappy chair (like most companies do on your behalf) is rather like being a professional driver and chosing a Daewoo Lacetti.

4. An Editor that doesn't mean you have to code with one hand on the mouse. Again, I think something Jeff blogged ago a while back, but seriously use something proper like MacVim or Aquamacs Emacs , once you get over the slight learning hump you'll find yourself being much more productive. Note: I'm not necessarily saying that you should ditch the IDE either, there are ways of using the both interoperably and for the tasks they're best at. While you're at it, it's worth spending a few minutes each day leaning the keyboard shortcuts for _everything_ on your OS of choice :)

5. Some completely awesome music to listen to. OK, not essential, but a nice to have. I'm going to go out on a limb here and recommend the new Negativland album Thigmotactic.

© 2007 Wired Up And Fired Up