E MICHAEL WITE RESPONDS:

It has been demonstrated that 3 brick layers can build a 30 foot high brick house , as shown here, in just 3 months. That's 4 a year.

Now if you had just 30 brick layers, they could build 40 per year, 80 in just 2 years. ​
Now that would be based on a 8 hour working day, five days per week.

But we know that back in the days of the kings, no king would allow such a slow pace to occur. ​​​
Instead you would expect at least  a 16 hour day and 7 days a week, resulting in twice as many structures, 160 in just 2 years.

Double the crew to a mere 60 brick layers and you would have laid enough bricks to build a structure taller than the tallest building in the world today.

As for the building of the city, we are told that it ceased with the confounding of the languages​ and dispersion of the people. ​​