An irrevocable trust is not an incontestable trust. If a court recognizes you as having standing -- a stake in the actions of the trust -- you can contest an irrevocable trust just as if it were a revocable trust or a will. If you prove your case, a judge may agree to terminate the trust or invalidate some of the terms.
A grantor -- the trust creator, sometimes called a settlor -- cannot create a valid trust if he's not in his right mind. If you can prove that the grantor wasn't mentally competent to manage his affairs, a court may set the trust aside. The same applies in situations of "undue influence": If you can show that the grantor wrote the trust provisions because someone intimidated him or persuaded him through fraud, your attack on the trust may succeed.
Each state sets up rules for creating and managing a trust. If you plan to contest your mother's trust, for example, and can prove that it doesn't meet state law, that would be grounds for invalidating it. In California, for example, a trust isn't valid if it doesn't name a beneficiary. Louisiana grants children rights to a percentage of their parents' estates and won't allow a trust to deprive them of their shares. Check the state law, and you may have a case.
If you want to contest a trust or will, you have to have standing; the law has to consider you someone with a legitimate interest in how the trust assets are managed and distributed. Trust beneficiaries have standing and usually so do the deceased's spouse, children and heirs. Creditors of the trust have standing. If you're not sure whether you have standing or not, check your state probate code or talk to an attorney.
Just because you don't like the terms or didn't inherit what you expected doesn't make the terms unfair and isn't proof of mental incapacity. Courts usually assume that the grantor is competent to dispose of her property as she chooses, so the burden of proving otherwise is on you. You should also check whether the trust includes a no-contest rule that cuts off beneficiaries who challenge the provisions. Also, keep in mind that if you contest what another family member receives from the trust, it could lead to years of hostility.