that said

introduces a contrasting or qualifying point

“The book is brilliant. That said, it's not for everyone.”

Origin: Short for 'having said that'