The .s4ext is describing build dependencies of the extension, but more specifically the other extensions that it depends upon. @jcfr and @lassoan can confirm, but this comment in the .s4ext should probably be clarified to list the dependent extension names rather than the name of dependent modules.
Your extension contains modules and for a given module you will want to specify dependent modules in the same type section as shown below. This makes sure certain modules are loaded first during startup so your module can appropriately import various things it might need during initialization. As in this linked example, the Segment Editor module depends on the Segmentations module and Subject Hierarchy module. In your case, your module would depend on the Markups module.