It's not even about the EPA, refrigerant is horrible, horrible stuff to inhale accidentally, if you were to take a good breath full of it you'd probably have some problems.
Your sure you're NOT thinking about the pressure or cold injury hazards? The biggest hazard I've heard credibly warned is if it sprays into the eyes in liquid form it could potential blind from the resulting cold injuries, even that sound remote to me, but I would never risk it. R-134a has a low toxicity if inhaled, as do most refrigerants.
I'm NOT an expert, but I see people warn about breathing in refrigerant, but there is no literature out there warning against it, and everything that talks about the toxicity of refrigerants states its a very low toxicity. The MSDS even says its low toxicity, but warns that replacing the oxygen in the lungs with it can cause heart arrhythmia.
Filling your lungs with any heavier than air gas, or anything other than air, has the potential to kill and could likely cause heart and lung problems, maybe that is where this "refrigerant is Nasty Stuff" comes from. I think its really just the hazard of dealing with any non-oxygen gas under pressure.
If you do something really dumb, let a freak accident happen, and fill your lungs with it, replacing the oxygen, you can cause problems. Think about it, if you did the same with water, you'd have all sorts of problems, but water isn't Nasty stuff is it?