low person on the totem pole

“I was the low person on the totem pole,” he said.

The white American male is not "the low person on the totem pole".

“The white man is a low person on the totem pole. Everybody else is above the white man.”

You’re also the new, low person on the totem pole.