This is a very unusual case of an apparently completely healthy man with a normal immune system (the system in our bodies that fights infection) who accidentaly inhaled a cloud of fungal spores while opening bags of compost. The case was so unusual it has been recorded in detail in the medical journal The Lancet. The article was reported in the UK press here.
The huge number of spores inhaled penetrated deep into his lungs as he breathed them in while he worked. This seems to have completely overwhelmed his bodies normal defences and many of the fungal spores grew quickly. He died within days from invasive aspergillosis.
It must be emphasised that this is a very rare case - the only other similar case I have heard of were of a farmer inhaling a cloud of spores from mouldy grain poorly stored in a silo. Nontheless it is important to highlight the importance of awareness of this risk as gardening is a hugely popular activity across the world - take care to avoid breathing in clouds of spores when disturbing decomposing plant material!