The 38-year-old politician said it was a hard decision, but that he was going ahead with the plan to pave the way for other male employees in his ministry and working fathers elsewhere.
Although Koizumi, 38, is only taking two weeks off, his decision is significant in Japan despite it having one of the world's most generous time off allowances for new fathers.
"Data shows that 80 percent of men, upon joining the work force, say they'd like to take paternity leave, but only 6 percent of them actually go through with it", he said in a speech, which was also posted on his website.
With Japan facing an ageing population and a dwindling birthrate, the government recently began promoting paternity leave.
"I hope my taking paternity leave will lead the way of working styles to one where everyone can easily take child-care leave without hesitation in the environment ministry", Koizumi told his staff, according to the Times.
The move is also notable as Koizumi - the son of former Prime Minister Junichiro Koizumi - is widely seen as a future prime ministerial candidate.
He has been closely scrutinised as a potential rising star in the government, with his comments and behaviour subject to intense media dissection.
That same study revealed Deloitte Australia topped the list for best workplaces for new dads, offering 18 weeks' paid parental leave with each employee able to take the time off over a three-year window.
Both parents can take up to a year off, with additional renewable six-month periods if a nursery place is unavailable. He said drew his inspiration from Mie Gov. Eikei Suzuki, who took parental leave while in office.
A handful of men have sued their employers alleging they were subject to what is known in Japan as "pata-hara", short for paternity harassment, after taking parental leave. He hopes to set an example for other fathers as the nation deals with a crisis of work-life balance, and declining birth rates.