I struggle with the argument over reparations for slavery. A formal apology from the British government seems a small concession, and I don’t want people already struggling with rent and bills to pay for crimes committed before their great-grandparents were born. I would press Andy Burnham towards debt relief and ask what happened to the fortunes made from slavery. Some of that wealth survived, and the families and banks that retained it have questions to answer. Yet I can hear how easily my objection to making taxpayers pay could become an excuse for the British state to absolve itself. I can’t accept that either.

Robert Tombs’s argument in the Telegraph makes me want to be more precise. He accuses Commonwealth governments of laying a trap and Burnham of declaring “open season on the British taxpayer”. Before we have established who might owe what, he invites us to imagine money disappearing from our own pockets. The owner of inherited millions and the worker whose wages barely cover the rent become fellow victims. An inquiry into wealth becomes a threat to everyone.

Which taxpayer should pay?

I had used the phrase myself: the British taxpayer shouldn’t pay. But a care worker and someone living off an inherited estate do not have the same resources or the same interest in protecting wealth. A government could tax large fortunes to finance repair, or make working households bear the cost through taxes on spending and cuts to services. Calling both a burden on the taxpayer hides the choice. I mean that working people shouldn’t pay, and I weaken my argument if I let Tombs fold them into a national interest defined by those with the most to protect.

Foreign Secretary Ed Miliband has supplied another reason to be sceptical. Asked on BBC Newscast whether the government’s approach could cost taxpayers anything more, he said no and ruled out a change of policy on reparations. The £40 million announced for Caribbean infrastructure comes from existing budgets, so it doesn’t represent additional spending. That weakens Tombs’s warning about a raid on our pockets. But Miliband’s reassurance also sets the limits of the government’s offer before its talks with Commonwealth countries have concluded. He rules out financial reparations without addressing whether banks or holders of inherited wealth should contribute.

Miliband dismisses Blair’s handling of the issue as “sorry, not sorry”. Yet Blair’s November 2006 statement, written for the black community newspaper New Nation ahead of the abolition bicentenary, acknowledged that “Britain’s rise to global pre-eminence was partially dependent on a system of colonial slave labour”. Blair then turned to development assistance. “Debt relief is already beginning to flow,” he wrote. Nearly twenty years later, much of the supposedly new approach is recognisable.

Miliband’s empty promise

Blair invited his readers to rejoice in the “different and better times” they inhabited and presented Britain as a country helping Africa through aid and international leadership. He didn’t ask what obligations fell on those who still held wealth derived from slavery. I understand why a formal apology would matter to people whose history has been denied, but Miliband needs to explain what will change beyond the wording. His promise that Britain will pay nothing extra doesn’t answer that question.

Tombs would prefer to close the historical argument altogether. He invokes electricity and computers as though later inventions could tell us where earlier capital came from or how its owners passed on their advantages. Slavery need not explain every subsequent source of British wealth to have contributed substantially to its growth. Research by Stephan Heblich, Stephen Redding and Hans-Joachim Voth finds that slavery wealth accelerated British industrialisation. Historians continue to dispute its importance, but Tombs can’t claim that their research has settled the question in his favour. Blair’s acknowledgement of partial dependence was more careful than Tombs’s attempt to dismiss the connection.

Nor does the small proportion of the population who owned slaves establish that their fortunes were insignificant. Concentrated ownership is what I would expect to find. Tombs moves from the few who owned to the unfairness of charging everyone, passing over the people who hold the wealth now. They are precisely the people I want to ask about.

Slavers were compensated while slaves got nothing

The British state has already shown how much it was prepared to pay when slave owners faced a loss. The settlement accompanying the 1833 abolition legislation provided £20 million in compensation to owners, whose financial interests were represented in Parliament. The enslaved received nothing, and many had to continue working without wages under the apprenticeship system. The state accepted an obligation to compensate people for losing their ownership of other human beings. It accepted no equivalent obligation to those whose lives and labour they had taken. Anyone defending the taxpayer today should have something to say about that decision.

Tombs’s invitation to compare Barbados with Haiti makes the omission harder to overlook. He offers the comparison as evidence of what British colonial rulers did for the societies they governed. But in 1825 France made recognition of Haitian independence conditional on an indemnity to former colonial owners. The indemnity and associated borrowing burdened Haiti for generations. People who had won their freedom were made to compensate those who had claimed them as property. Tombs invokes Haiti’s suffering while leaving out the financial demands that helped produce it.

Tracking where the slavers’ wealth ended up

Some of the proceeds of slavery are still visible in Britain. Harewood’s own account explains that the Lascelles family used profits from trafficking enslaved people to purchase the Harewood and Gawthorpe estates. The wealth became property which remains with us. UCL’s research has traced the commercial and institutional legacies of slave ownership much more widely. Two centuries of inheritance and investment complicate the task, but they don’t make it absurd. A family tree alone proves neither present benefit nor personal guilt; we need to establish what was inherited and who controls it, including property held through trusts. A wealthy family cannot celebrate generations of ownership when asserting its entitlement to an estate, then treat that history as irrelevant when someone asks how it acquired the land. We need evidence of the benefit, and we should not have to wait for an heir to volunteer it.

Banks belong in the inquiry too. Research published by the Bank of England shows how intermediaries earned commissions from collecting slave-owner compensation. The largest agents included partners in London banks and merchant firms with existing colonial connections. Owning a plantation was only one way to profit. Following the money through the firms that inherited these businesses will be harder than naming a few aristocrats, but stopping at the country-house gates would leave too much untouched.

The government should require families and institutions to open relevant records, and fund the work of tracing the assets. People in the societies seeking restitution should help decide what it achieves. Voluntary philanthropy leaves the holder of inherited wealth in charge of how much to give and where the money goes. A scholarship fund may do useful work, but the donor can’t decide that it settles the claim.

That investigation mustn’t become a way for ministers to pass responsibility to private families. The state upheld slave ownership and organised compensation for its owners. The officials are dead; the institution survived. Nor should action depend on tracing the origins of every pound. The government can accept responsibility for what the state did and raise money from large fortunes without claiming that everyone living in Britain shares an inherited guilt. Where the records cannot establish a direct beneficiary, the case for public action remains.

Debt cancellation is an option

Blair’s statement also makes me reconsider my preference for debt relief. I can’t offer it as the answer to an empty apology when his government was already proposing that combination over twenty years ago. I would still press Burnham towards it, because lower debt payments could release money for public services, but I want to know what borrowers will actually gain. His proposal involves longer repayment periods and better interest rates. Those changes could help. They could also make debts easier to collect without greatly reducing what countries must pay. Extending a loan may avert an immediate crisis while leaving a country in debt for years longer.

I would demand a substantial reduction in what creditors can recover. Britain should press for cancellation where debts are unsustainable, with private creditors accepting losses instead of being bought out at public expense. Writing off debts owed to the British government would have a cost too. Supporting relief doesn’t let me evade that: I would place the cost on wealth rather than wages, and oppose privatisation or further cuts to public services as conditions of an agreement. A scheme that protects creditors’ returns while giving borrowers a little breathing space would fall short.

There is a further difficulty with treating relief as an alternative to reparations: debt cancellation has long been part of CARICOM’s reparations programme. I can’t welcome the substance while treating the word reparations as proof of an unreasonable demand. The Caribbean governments’ programme connects debt with public health and development. My reservations have to address those proposals, rather than the demand for an unlimited cheque that Tombs attributes to them.

State and rich should pay not workers

I don’t want Britain’s poor set against the poor of its former colonies while the owners of wealth watch from a comfortable distance. Nor should debt relief give British officials another means of supervising the countries receiving it. People affected must control how the resources released are used, with their trade unions and public institutions able to hold governments to account. Decisions taken elsewhere and presented as generosity would leave too much of the old relationship in place.

My unease has not disappeared. I would oppose another squeeze on working people, but Miliband’s promise that an apology will cost nothing does not satisfy me either. He leaves inherited fortunes and creditors’ claims unexamined. I would push Burnham towards debt cancellation and demand an account from the banks and families whose wealth can be traced to slavery. The government should fund and organise that work while negotiating repair with the societies affected. Opposing a bill for working people obliges me to pursue those who can and should pay. If I leave their wealth untouched, I am helping Tombs defend it.