Handlebars provide 0 attribute with current index in the each iteration.
You can write a custom helper and use current index to calculate the current cell.
<td data-cell-type="formula">=A{{currentCell 0}}</td>
function currentCell(index) { return index + 1 }