Large language models can provide consistent and reliable annotations of argument quality, potentially enhancing and streamlining human-led efforts in this domain.