The company said it had no plans to extend the network into the countryside, concentrating instead on expansion in towns and cities.
The decision could "exacerbate the rural-urban digital divide", said the Country and Land Association (CLA).
But a group that represents rural broadband providers was unsurprised.
"With ?1.7bn of public funding going almost entirely through BT, it isn't surprising that an investment of this scale is focusing on areas that Virgin expects to see delivering better results," said Malcolm Corbett of the Independent Networks Co-operative Association.
Virgin Media currently has five million customers and its latest investment, which it claims is the biggest in broadband infrastructure in the UK for more than a decade, is expected to extend its network from 13 million to 17 million homes.'Look beyond cities'
The CLA acknowledged it was not always profitable for private businesses to invest in remote areas.
However, a spokesman said it was "a pity that private companies are not putting the infrastructure in place to benefit rural business".
"What is clear is that rural areas will not be able to access speeds in excess of 30Mbps or 50Mbps for some time to come. All this does is show that the rural economy is neglected once again," he added.
Many rural business and families would like to be able take advantage of Virgin Media's Fibre to the Home technology, he said.
"BT's rollout of superfast broadband under the government's rural broadband programme relies on fibre to the cabinet (FTTC). This further disadvantages rural communities and businesses from effective connection speeds, because the further away a business is from the cabinet, the slower the speed."
Meanwhile the Countryside Alliance's head of policy, Sarah Lee, said it was "disappointing that the improvements will not be seen in rural areas, but we welcome this challenge to BT's monopoly".
"We hope this investment will encourage all broadband providers to look beyond the cityscape and invest in our countryside communities, to enable them to compete economically and socially in this digital world."