Reply to Sample rights

What is the legal or acceptable limit that you can sample a song or a small part of a vocal/acapella? Like is a maximum 3-5 second vocal rip ok even without getting permission from the original singer etc.?

I know the general rule of thumb is to ask the original, but that is sometimes a tough process so I'm just wondering.