Its really hard to measure effectiveness, problem becomes even harder when a non-engineering person has the job to measure effectiveness of a engineering person.

On other hand. for software engineering some of the signals that can be used to measure such a management itself can be

1. On call requirement, outages and team burnout - A well written software should not require on-calls from the dev team

2. Ask them about the "concrete" roadmap for next 6 months to a year - Absence of concrete items is a bad sign