Even more ideal would be using composer packages for the language packs of each theme/plugin, currently only those are only available for the most popular plugins/core themes, see Install&Update wordpress languages with composer.
Mabye one could solve this by implementing new composer types which will query the WordPress API directly? It would avoid pulling all possible permutations of plugin/theme version + language pack versions.