I'll often start from an "intellectual" idea.. some concept that I would like to develop, like a specific interval,or combination, or a rhythm, or when writing for a specific instrument some element that would only be playable on that particular instrument.
Once I've settled on that aspect, I might let my hands fall on the keyboard and see what comes from that, or explore some of the basic harmonic techniques that are part of my harmonic language.
I've also resorted to the "reverse engineering" approach at least once - in my 2nd symphony, the final movement, I took what was a recurring 5-note motif throughout the symphony, and from that actually filled out the missing seven notes to make a tone row, then used the material in a very loosely structured way as two halves of a single theme (one 7-notes long, the other 5-notes long).