Purpose: Data envelopment analysis (DEA) and analytical hierarchy process (AHP) are two widely applied methods to evaluate and rank suppliers in terms of sustainability. In this study, to investigate the extent to which potential differences in the outcomes of these two methods influence the benchmarking strategies, a comparative analysis based on a common set of data gathered from 19 logistics service providers is implemented.

Design/methodology/approach: As suppliers' sustainability cannot be improved in a single-step process due to several limitations, improvement needs to proceed gradually. Therefore, using the self-organising map method, the suppliers were classified into clusters within a novel framework for gradually improving their sustainability. Then, the two processes of gradual improvement based on the outcomes of DEA and AHP were compared.

Findings: The findings show that although the rankings of suppliers guided by the methods correlated to a high degree, the benchmarking strategies provided by the methods for gradually improving the sustainability of suppliers differed considerably. In particular, whereas AHP suggests a benchmarking policy better suited for unsustainable or less sustainable suppliers with limited access to resources, DEA proposes one for suppliers able to dramatically boost their sustainability with few quick, significant leaps in performance.

Originality/value: First, this study revealed a novel gradual improvement framework using the self-organising map method. Second, it clarified the extent to which the benchmarking policies are influenced by the type of evaluation method.