A separate report showed a decline in private-sector employment, down to 119,000 in April, which is primarily being blamed on tax hikes and the sequester, according to the ADP National Employment report released earlier Wednesday, which that measures private-sector job growth.
Companies pulled back on stocking up their shelves, with the inventories index falling to 46.5 from the 49.5 reported in March.
Despite the slow down, the index showed a bump in new orders, up to 52.3 while production rose to 53.5 from 52.2.
The Labor Department is set to release its measure of private and public job creation on Friday.
The slower activity in the manufacturing sector may be reflecting concern about the effects of $85 billion in across-the-board spending cuts that kicked in on March 1.
The ADP report showed that manufacturing employment fell by 10,000 in April.
Still, consumers have remained resilient despite the elimination of the payroll tax cut, spending at the fastest clip in two years through the January-March quarter.
But consumers started to show signs of spending fatigue last month, a possible signal that they are beginning to feel the effects of smaller paychecks.
Still, they are generally remaining confident in the economy and hiring, which could keep the recovery chugging along through the spending cuts.
In another report on Wednesday, the Commerce Department said construction spending dropped 1.7 percent in March, dragged down by a 4.1 percent cut in government spending, the largest decline in more than a decade.
Still, construction spending is up 4.8 percent over the same period last year and has been helped by pick ups in residential construction, which increased 0.4 percent in March.