Robert Jenrick says the country could not afford to "help working families have more children".