Cell borders in xlsx recipe
How to set boundaries for a cell (lower, upper, all) in the xlsx recipe?
Don't know it from the top of my head, but you can easily find it out.
Create simple xlsx in your desktop excel with the boundary you like.
Then save it, rename it to the zip and inspect the xml how is it defined.
Then replicate the same using xlsx helpers.