Untick the "show column menu" checkbox in the "Display As" property of the query
I think this removes only the three dots in the column header. Jannes probably refers to the three dots left of the icons in the rows.
@Jannes, did you untick the 'display operation name on the button' option of the settings of the row operations? (and therefore only the icon is ticked)