First, recognize that breaking objects is about a few very simple ideas:
- proper positioning of the striking member: Different materials and objects have different characteristics that can simplify or complicate their breaking. The most common example is wood boards - you want the striking member to be aligned to the grain of the wood (e.g. a side kick orients the foot parallel to the floor, therefore the grain should run parallel to the floor as well). It's possible to break an object with improper orientation, but it can be significantly more difficult and dangerous.
- speed at the moment of impact: The speed of the striking member at the point of impact is key in the breaking of an object. Most people have difficulty with this idea. It's best seen during speed breaks where an unsupported object is broken through speed more than power (as the lack of support mitigates the influence of power, placing more emphasis on the impulse at the point of break). Put more bluntly, you can break with speed and you can break with raw force, but you'll get the best effect when you use both. More to the point, when you start trying to break more complex objects like broom handles, speed becomes increasingly important - hesitance is your enemy.
- raw force applied to the object: This is something that most people intuitively grasp - the more force you apply to an object, the easier it is to break. This is true, but without speed, you'll generally find breaking more difficult than necessary. That said, few things substitute for pure, brute force - particularly when breaking more dense materials like concrete shingles.
- (in some cases) conditioning of the striking member: The title of your post talks about breaking with a fist. First, I want to warn you that this is significantly more painful than you may suspect. In fact, a break with the knuckles of the fist is something that my organization reserves for demos by people who've put in the time for the appropriate conditioning. Most breaks on most materials don't really require a lot of conditionging beyond that nearly all martial artists perform as part of the routine of normal training. This changes when you start dealing with odd objects (broom handles, cocunuts, etc) and odd striking members (shins, head, etc). For these situations, you should start with simple breaks - objects with greatly reduced resistance. Combined with conditioning the striking member against traditional conditioning materials (striking heavy bags unprotected, striking sand, etc), you can build both conditioning and confidence. In some cases (such as concrete), conditioning is insufficient for some and additional protection (sugical or duct tape is often sufficient) is required in order to avoid scraping that could result in bleeding. Don't overestimate the importance of confidence - a lack of it will result in pain more often than not.
If you're not already trained in the basics of breaking (including what I mean by these four concepts), then that's your first step. I'd strongly suggest that you practice these basics with an instructor - they'll best be able to help identify any issues with your application of these ideas.
You asked two other questions that I'd like to tackle. The first is the question of how to break wood. Like I mentioned before, wood is most easily broken when breaking with the grain. Thickness and knotting also play important roles - the thicker the board and the more knotting, the more difficult it will be to break. However, the type of wood is strikingly important. White pine is what most marital artists use - it's breakable, common and easy to find without knots (though it's difficulty ramps up when damp). Other types of wood can be significantly more difficult to break. With something like a broom handle, this is almost certain to be something other than pine - ash and hickory are more common and far more difficult to break than white pine.
The second question is how to break without hurting yourself. To be honest, you really have to work up to it. There is no way to know before starting that you're not going to hurt yourself. This is why you need to learn technique first and work your way up from more traditional, safer materials (rebreakable boards or white pine boards cut for breaking).
To be clear, board breaking as a basic act isn't difficult - most martial artists could teach your average person to do a basic single board break with 15-30 minutes of practice and a conducive board (white pine, 11", grain lined up properly, well braced). It gets significantly trickier when you start moving up to stacked (non-spaced) boards, odd striking members (heads, shins, knees), more complex striking materials (concrete, brick, ice) and more irregular objects (broom handles, coconuts). You seem to be talking about breaking a broom handle with a fist (I would assume a hammer fist - I seroiusly hope you're not considering a punch break) - this is a difficult break for a few different reasons. Broom handles are made out of fairly sturdy wood, you're breaking against the grain and you're using a striking implement that can be difficult to use properly without practice and training (many adults avoid hammerfist breaks because of the occasional bad aim resulting in the break missing the meat of the palm and hitting with the bones of the wrist instead - those break in extraordinarily painful ways...). Before attempting such a break I would strongly suggest working with an instructor on a plan to build up to the break before attempting it. Its a great demo piece (though its usually done with an elbow or shin in my experience), but as with most great demo pieces, not something you want to try without the appropriate practice to build you up to it.