"CSS layout" needs defined before anyone can properly answer this.

Since it's a vague question I'm going to give a vague answer :p

AFAIK the first forum default theme to not have any non-semantical tabels was bbPress. It does use tables, but they're semantical tables. So the answer you are looking for is probably bbPress I'm guessing ... but then perhaps not if you are defining "CSS layout" as something which has no tables.

phpBB3 was the first software to provide a default theme which used no tables at all. But just because something uses a table to define tabular data, does not mean that the layout is not defined by CSS, therefore being a "CSS layout".

However there is also a third option in that I'm sure there must have been a third party forum theme out there which used no tables at all long before bbPress, phpBB3 or Vanilla released their semantically optomised default themes. I have no idea what the first one of those would have been though.

And there is a fourth option ... themes which used convoluted nested tables on the interior content of their pages but use semantical non-table based layouts to control the actual layout of the page as a whole, but not for the nitty gritty stuff like positioning off avatars, posts, signatures etc.

If I had to pick just one option, I'd go for bbPress since AFAIK it was the first forum software to provide a semanticaly optomised theme by default.