No your not missing something, it’s not possible to delete records directly from an updateable dashboard.
A way that I found that worked was to add a checkbox to the dashboard called ‘Delete Row’
and then modify the update directive, using the following logic where the custom code deletes the row