We’ve all heard the saying before circa 17-something: “nothing is sure in life but death and taxes.” However true this may be, taxes aren’t the same when you choose to retire, and many new retirees aren’t prepared and don’t completely understand in what ways their finances will change at this new juncture in their life. Some individuals are confused about the process, wondering if taxes will be easier or harder now that they’re no longer working.

The answer will differ according to your income bracket and place of residence, however some tax structures apply to almost everyone. The first thing you should know is that you may still have to pay taxes during retirement, even if your income will drop significantly depending on your source of income. The most important way to prepare is to calculate your tax responsibility ahead of time, so you’re not caught off guard.

Different incomes mean different tax rules. Understand these various forms of income so you can determine how many of them you’ll have and how much money it totals. We’ll begin with the most common forms of income.

Social Security

You’ll receive Social Security from the federal government, and if you are some of the millions of Americans who only rely on Social Security to live during their retirement, you likely won’t pay a penny of tax. There are 37 states that don’t tax Social Security, so if you reside here and Social Security is your only form of income, it’s time to rejoice! However, if you’ll also receive income from other sources, then you will be taxed on your Social Security. Your additional income (known as combined income) will determine how much of your Social Security is taxed, from 0–85%. The tax worksheet from the federal government will help you determine this exact amount.

If you have a large amount of money coming from retirement accounts, investments, etc, then you can expect to be taxed on the higher end of your social security. If you have little to nothing coming in outside of Social Security, you’ll be on the lower end of the tax burden. For married couples, incomes upwards of $44,000, and incomes over $34,000 for single retirees will warrant the most taxation. Individuals earning less than $25,000 and couples bringing in under $32,000 will be tax-free.

If this is the case, and you don’t have to pay taxes on your Social Security, then you probably won’t have to file a tax return anymore! For many retirees with conservative incomes, the age of filing yearly returns is over.

IRAs and 401(k)

If you have an IRA or 401(k), you will be required to pay some taxes. The amount you withdraw from these accounts yearly counts as your taxable income. Your taxes will also largely depend on the amount of deductions you have for that year (say you have several medical expenses this year) and the tax bracket you’re in.

Another important thing to consider if that Roth IRA withdrawals are tax-free most of the time. Roth IRAs are individual accounts and let workers designate a portion of his/her after-tax income yearly. Your earnings and withdrawals are tax-free after 59.5 years old.

Pensions

When it comes to pensions, if you deposited the funds before taxes, then you will be taxed on that money when it’s time to withdraw it. Most retirees’ pensions are funded from pre-tax income, which means it will be counted as taxable income each year. You can choose instead to have taxes withheld from your pension check so Uncle Sam is taken care of directly.

If your pension was funded by some pre-tax income and some post-tax income, then you will have some withdrawals that are taxed and some that are not in proportion to how the account was funded.

Annuities & Investments

Your tax rules regarding annuities will depend on whether it’s owned by an IRA or purchased with after-tax dollars. If you have an IRA-owned annuity then normal IRA tax rules will apply. Conversely, if your annuity was not purchased in an IRA or retirement account, your tax rules are based on the kind of annuity you have: namely, immediate, fixed or variable.

Immediate annuity: Only the portion of the annuity that is considered interest will be taxed. Your company will tell you what this amount is each year.

Fixed or Variable: Your earnings (if the amount of your accounts is greater than what you originally contributed) must be withdrawn first, which will be taxable income. After you’ve withdrawn your earnings, you’re left with your initial contributions, which aren’t counted as taxable income.

Keep in mind that this is just an overview of what taxes in retirement will look like for the average American household. Your tax situation may look different depending on the income available to you and how you choose to withdraw and allocate your funds.