I think it's mostly in how clean the fire is and how much smoke. Dirty smoke with creosote will definitely have a bad aftertaste and furthermore you'll keep burping it up.
I agree with Dustin and would add to make sure the type of wood is matched to the product you are smoking. Strong works like mesquite will over power poultry and give you what some would refer to as an after taste.