How to go from generalist to specialist?

Thanks for some of the great comments in recent days encouraging specialization in the practice of law. Going a step or two further, and I'm sort of describing the debate that's going on inside my head rather than suggesting I have an answer, what if you are too much of a lawyer generalist...how do you transition toward being that desired expert/specialist?

I'd guess that my journey towards sole practitioner and arguably too much of a generalist, is fairly common. When my solo firm was started some 2+ years ago I had practiced in 3-4 practice areas primarily based on the practice areas of the two employers I'd worked for previously. So those 3-4 practice areas were what I knew (to some degree) and that along with the need to "pay the bills" led me to probably be a bit too general. We're say 60% various forms of family law, 20% residential real estate, 10% trusts/estates and 10% landlord-related matters. I certainly turn down cases where I'm not competent but at the same time there certainly are situations where I feel like my knowledge isn't specific enough in some of the nuances of our practice.

If you have a practice like mine or even a more general solo practice with maybe some corporate work too, how do you choose which area becomes the "new" speciality? What drives the choice? Your interests? Market-based analysis? Both? Other factors?