The overall quality of the study, as assessed by the Amstar 2.0 tool, shows mixed results across different domains.

Strengths:

PICO criteria (Domain 1): All PICO elements are present and well-described in the review.

Description of included studies (Domain 8): The review provided adequate detail about the included studies, following the PICOS framework and presenting results in a comprehensive manner.

Appropriate methods for statistical combination of results (Domain 11): The authors used appropriate methods for the statistical combination of results, but there was high heterogeneity among the studies.

Weaknesses:

Explanation of study design selection (Domain 3): Although study types were clearly stated, no justification was provided for their inclusion or exclusion.

Study selection in duplicate (Domain 5) and Data extraction in duplicate (Domain 6): It’s unclear if two reviewers performed these tasks in duplicate or if disagreements were resolved by consensus.

List of excluded studies (Domain 7): There is no clear indication whether a list of excluded studies was provided or if the reasons for exclusion were given.

Risk of Bias assessment (Domain 9): It is not clear if a formal risk of bias assessment was performed, if this assessment was done by independent assessors, or if the results were used in the synthesis.

Funding sources for the studies (Domain 10): The authors did not report on the sources of funding for the studies included in the review.

Assessment of RoB in individual studies (Domain 12) and Accounting for RoB when interpreting/discussing the results (Domain 13): The authors did not explicitly mention that they assessed the potential impact of risk of bias in individual studies on the results of the meta-analysis, nor did they state whether they accounted for the risk of bias in the individual studies when interpreting or discussing the results of the review.

Explanation for heterogeneity (Domain 14): The authors recognized and reported on the heterogeneity in their results, but they did not provide a detailed discussion of its sources.

Investigation of publication bias (Domain 15): The authors did not perform an adequate investigation of publication bias.

In summary, while the study has some strengths in terms of PICO criteria and description of included studies, it has several weaknesses, particularly in terms of bias assessment, study selection, data extraction, and reporting of potential conflicts of interest. These weaknesses could potentially impact the reliability and validity of the study’s findings.

Summary table with Amstar 2.0 assessment

Domain	Assessment	Justification
Domain 1: PICO criteria	Yes	All PICO elements are present and well-described in the review.
Domain 2: Established methods prior to review	Partial Yes	Although PRISMA guidelines were followed, there was no explicit statement about a written protocol or justifications for deviations.
Domain 3: Explanation of study design selection	No	Although study types were clearly stated, no justification was provided for their inclusion or exclusion.
Domain 4: Comprehensive literature search	Partial Yes	A search strategy was partially followed, but certain aspects such as multiple database searches, MeSH terms, and inclusion of grey literature were missing or inadequately described.
Domain 5: Study selection in duplicate	No	Although two reviewers were involved in the selection process, it’s unclear how disagreements were resolved.
Domain 6: Data extraction in duplicate	No	It’s unclear if two reviewers performed data extraction in duplicate or if disagreements were resolved by consensus.
Domain 7: List of excluded studies	No	There is no clear indication whether a list of excluded studies was provided or if the reasons for exclusion were given.
Domain 8: Description of included studies	Yes	The review provided adequate detail about the included studies, following the PICOS framework and presenting results in a comprehensive manner.
Domain 9: Risk of Bias assessment	No	It is not clear if a formal risk of bias assessment was performed, if this assessment was done by independent assessors, or if the results were used in the synthesis.
10: Funding sources for the studies	No	The authors did not report on the sources of funding for the studies included in the review.
11: Appropriate methods for statistical combination of results	Yes	The authors used appropriate methods for the statistical combination of results, but there was high heterogeneity among the studies.
12: Assessment of RoB in individual studies	No	The authors did not explicitly mention that they assessed the potential impact of risk of bias in individual studies on the results of the meta-analysis.
13: Accounting for RoB when interpreting/discussing the results	No	The authors did not explicitly state whether they accounted for the risk of bias in the individual studies when interpreting or discussing the results of the review.
14: Explanation for heterogeneity	No	The authors recognized and reported on the heterogeneity in their results, but they did not provide a detailed discussion of its sources.
15: Investigation of publication bias	No	The authors did not perform an adequate investigation of publication bias.
16: Reporting of potential sources of conflict of interest	Yes	The authors did not reported no conflicts of interest.

Detailed description of assessment

Domain 1: Did the research questions and inclusion criteria for the review include the components of PICO?

Based on the provided text, the systematic review does indeed appear to incorporate the four components of the PICO framework:

Population (P): This is clearly described in the “Selection of studies” section as patients aged over 18 years with a diagnosis of type 2 diabetes mellitus (T2DM).

Intervention (I): The intervention in this review, which is actually more of an exposure, is the measurement of HbA1c variability, assessed by the standard deviation (SD) or coefficient of variation (CV). This is described in the “Investigated condition” subsection of “Selection of studies.”

Control (C): The control condition is defined in the “Comparison condition” subsection as “logistic or Cox regression analysis for outcome risk prediction”. The language here is not typical for a systematic review of intervention studies. This likely represents the statistical methods used to adjust for potential confounding variables when assessing the relationship between HbA1c variability and outcomes in the included studies.

Outcomes (O): Outcomes are described in the “Outcome” subsection as the “risk of adverse macro-/micro-vascular complications” which are then elaborated in the next paragraph to include specific complications such as all-cause mortality, cardiovascular mortality, stroke, transient ischemic attack, coronary heart disease, myocardial infarction, peripheral arterial disease, nephropathy, neuropathy, and retinopathy.

So, the assessment of whether the research questions and inclusion criteria for the systematic review include the components of PICO is “Yes”, as all the elements are present and sufficiently described in the review.

Domain 2: Did the report of the review contain an explicit statement that the review methods were established prior to the conduct of the review and did the report justify any significant deviations from the protocol?

Assessment: Partial Yes

Justification:

The report of the systematic review provides some indication that the review methods were established prior to the conduct of the review. This is shown by the statement that the meta-analysis was conducted according to PRISMA Statement guidelines, following the suggested checklist of items. PRISMA is a recognized protocol for conducting systematic reviews and meta-analyses, and following this checklist suggests that the authors had established their methodology before conducting the review.

However, the authors did not explicitly state that a written protocol was followed or independently verified, nor is there any mention of registration with a platform like PROSPERO, submission to a research office, or a research ethics board.

Moreover, there is no justification provided for any significant deviations from the protocol. This implies that either there were no significant deviations from the established methods or that such deviations were not deemed worthy of mention or justification by the authors.

Therefore, based on the provided information, the report appears to only partially demonstrate the pre-establishment and adherence to a written and independently verified review protocol.

Domain 3: Did the review authors explain their selection of the study designs for inclusion in the review?

Based on the provided text, the authors of the systematic review did clearly explain their selection of study designs for inclusion in the review. They adhered to the PICOS (Population, Intervention, Comparison, Outcomes, and Study design) framework in defining their eligibility criteria. Regarding the “Study type,” the authors indicate they would include “any kinds of clinical trials (randomized controlled trial, cohort study, etc.)”. This statement suggests that they were open to both randomized and non-randomized study designs.

However, they did not provide an explicit justification for including non-randomized studies or restricting to randomized controlled trials (RCTs). While they included a range of study types, they did not explain why they chose to do so or why certain study types (e.g., reviews, editorials, case reports) were excluded.

Therefore, while they did follow a strategy in selecting study types, they did not adequately justify the inclusion of non-randomized studies or restriction to RCTs. Given this, the assessment of this domain should be “No”, because they did not explicitly justify the selection and exclusion of specific study types.

The criteria for the assessment have been partially fulfilled; the authors clearly explained their selection but did not adequately justify it. This missing information might have given more insight into their methodological choices, and its absence limits the transparency of their research methodology.

Domain 4: Did the review authors use a comprehensive literature search strategy?

Based on the information provided in the methodology section of the study, the authors partially adhered to a comprehensive literature search strategy.

The authors reported that they conducted their literature search on PubMed and specified the search terms used, which included key phrases and terms relevant to their study. However, they did not specify if they used MESH terms in their search, which is a commonly used method to enhance the specificity and sensitivity of literature searches. Moreover, the authors only mentioned one bibliographic database (PubMed), rather than searching at least two databases, which would have provided a more comprehensive coverage of the available literature.

The authors provided the time range of their search (2015–July 2022), and stated that they did not impose any language restrictions, which is good as it ensures the inclusion of relevant studies in all languages. The authors did not mention if they sought for grey literature, which might have led to a potential bias as grey literature can sometimes contain relevant information that is not found in published articles.

The authors mentioned that two reviewers reviewed all resulting articles, which is a good practice to ensure the robustness of the review process. However, the authors did not specify if they supplemented their searches by checking published reviews, specialized registers, consulting with experts in the field, or reviewing reference lists of included studies. These steps can further ensure that all relevant articles are included in the review.

In summary, although the authors did follow some aspects of a comprehensive literature search strategy, they did not fully adhere to all aspects of it. Therefore, the assessment for the authors’ literature search strategy is a “partial Yes”. The elements that were missing or not adequately described include: the use of MESH terms, search in more than one bibliographic database, search in grey literature, and the supplementation of search by checking published reviews, specialized registers, consulting with experts in the field, or reviewing reference lists.

Domain 5: Did the review authors perform study selection in duplicate?

Based on the provided text, it is evident that the authors performed study selection in duplicate. The text states: “All resulting articles were reviewed by two reviewers (R.C. and G.S.).” This indicates that two review authors were involved in determining the eligibility of studies for inclusion.

The text also provides information about the eligibility criteria which were set according to the PICOS (Population, Intervention, Comparison, Outcomes, and Study design) framework, and how each criterion was defined. It further states: “Full articles on potentially relevant studies were downloaded and reviewed for inclusion.”

However, the provided text does not explicitly mention the use of a consensus process in the case of disagreements between the two reviewers during study selection, or the use of a Kappa score to assess inter-reviewer agreement. This lack of specific information makes it unclear how conflicts or discrepancies in study selection were resolved.

Therefore, my assessment for this domain is “No,” because although two reviewers were involved in the study selection process, there is no explicit information about how disagreements during this process were resolved or if a second reviewer checked agreement on a sample of studies using a Kappa score.

Domain 6: Did the review authors perform data extraction in duplicate?

Based on the provided text, it appears that two independent reviewers (R.C. and G.S.) were involved in reviewing the articles resulting from the literature search as mentioned in the ‘Data sources and searches’ section. However, the text does not specify whether these reviewers also performed data extraction in duplicate, nor does it mention a consensus process for addressing disagreements. There’s no reference to a second reviewer checking the agreement on a sample of studies or the achievement of a Kappa score indicating ‘strong’ agreement (0.80 or greater).

Therefore, the assessment for this domain is ‘No’. While it is clear that two reviewers were involved in the review process, the specific steps of independent data extraction in duplicate, resolution of disagreements via consensus, and the usage of a Kappa score for evaluating agreement are not adequately described in the provided text.

Domain 7: Did the review authors provide a list of excluded studies and justify the exclusions?

Based on the provided text, it is not clear whether the authors of the systematic review provided a list of excluded studies and justified the exclusions. The text mentions that 23 studies were used for quantitative synthesis through meta-analysis, but it does not explicitly state whether a list of excluded studies was provided or if the reasons for exclusion were given for each study.

Therefore, the assessment for this domain would be “No” as the text does not provide information on excluded studies and their justifications.

It’s also not mentioned whether the list of excluded studies is available on request if it is too long to include in the report.

Domain 8: Did the review authors describe the included studies in adequate detail?

Assessment: Yes

The authors of this systematic review provided a detailed description of the included studies following the PRISMA guidelines. They clearly defined their population of interest (patients aged over 18 with a diagnosis of type 2 diabetes mellitus), the intervention being investigated (measurement of HbA1c variability), the comparison (logistic or Cox regression analysis for outcome risk prediction), the outcomes (risk of macro-/micro-vascular complications), and the types of studies included (any clinical trials). This approach is in line with the PICOS (Population, Intervention, Comparison, Outcomes, and Study Design) framework, which is recommended for systematic reviews.

The outcomes, including macrovascular complications (stroke, transient ischaemic attack, coronary heart disease, myocardial infarction, peripheral arterial disease) and microvascular complications (nephropathy, neuropathy, retinopathy), were clearly defined.

They also described their literature search strategy, selection process, and data analysis methods in sufficient detail. Their search terms, time range, and database (PubMed) were specified, along with the process of having two reviewers to evaluate the articles. They also explicitly described the statistical methods used for their meta-analysis.

The authors presented the results of their systematic review and meta-analysis in the form of text and tables, enhancing the comprehensibility of their work. The key characteristics of the included studies were summarized in a table, and the flow of study selection was demonstrated through a flowchart. Also, the results of the meta-analysis were graphically presented as forest plots.

However, while the authors did provide some detail on the study design (clinical trials), further specifics such as the locations or settings of the original studies were not provided in the presented text. Nonetheless, considering the overall information provided, the authors described the included studies in adequate detail, allowing the reader to assess the generalizability of the results.

Domain 9: Did the review authors use a satisfactory technique for assessing the risk of bias (RoB) in individual studies that were included in the review?

Based on the text provided, it is not possible to determine whether the authors of the systematic review used a satisfactory technique for assessing the risk of bias in the individual studies included in the review.

There is no mention in the provided text about the use of a formal tool to assess the risk of bias. Some commonly used formal tools to assess the risk of bias include the Cochrane Risk of Bias Tool, ROBINS-I, and the Newcastle-Ottawa Scale, among others. The use of such a tool would help ensure a systematic and comprehensive evaluation of the potential biases in the included studies.

Furthermore, there is no reporting of the results of the risk of bias assessment for each study. It is considered best practice to include a description of the risk of bias for each study in a systematic review, as this information is important for readers to evaluate the strength of the evidence presented.

There is also no information about whether the risk of bias assessment was performed by at least two independent assessors and whether a consensus process was used when disagreements arose. This process is important to reduce the potential for bias in the review process itself.

Finally, there is no indication whether the results of the risk of bias assessment were used appropriately in any subsequent synthesis. This step is critical as it influences the weight given to the results of each study in the meta-analysis.

Based on these observations, the assessment of whether the authors used a satisfactory technique for assessing the risk of bias would be “No”. Specifically, there are missing elements about whether a formal tool was used to assess risk of bias, the results of the risk of bias assessment for each study, whether the assessment was performed by at least two independent assessors, whether a consensus process was used when disagreements arose, and whether the results of the risk of bias assessment were used appropriately in the synthesis.

Domain 10: Did the review authors report on the sources of funding for the studies included in the review?

Based on the provided text, the authors of the systematic review did not report on the sources of funding for the studies included in the review. There is also no acknowledgement that the source of funding for a study can be associated with the direction and strength of the study results. Thus, the assessment for this domain is “No”. The authors did not mention the funding sources for the individual studies, nor did they discuss potential conflicts of interest that could arise from these funding sources. These elements are missing or not adequately described, which might be important for interpreting the validity and reliability of the research findings. It is essential to consider funding sources because they can influence the design, methodology, results, and interpretation of the studies, and therefore potentially bias the outcomes of the systematic review.

Domain 11: If meta-analysis was performed did the review authors use appropriate methods for statistical combination of results?

The authors of the systematic review appear to have used appropriate methods for the statistical combination of results in their meta-analysis. They have clearly stated and followed the PRISMA Statement guidelines for conducting meta-analyses, including their approach to study selection, data extraction, and analysis.

In terms of the statistical approach, they have utilized the random-effects model for the meta-analysis. This model is appropriate when there is a likelihood of substantial heterogeneity among the studies, as it considers both within-study and between-study variations, which is important when combining studies of different designs and contexts.

Regarding missing data, the paper does not directly address how the authors handled it, but since they strictly followed the PRISMA guidelines, one might infer that missing data was handled according to the accepted standards within those guidelines.

The authors stratified the analysis according to the presence of data regarding HbA1c variability in terms of coefficient of variation (CV) or as standard deviation (SD) and used hazard ratios (HR) or odds ratios (OR) data, as made available by considered studies. This stratification seems reasonable given the nature of the variables in question and helps make the findings more interpretable.

However, the authors noted substantial heterogeneity in their results, with I2 statistics indicating significant variability among studies, with ranges often greater than 50%. This high level of heterogeneity suggests that there is a considerable amount of variation among the effect sizes of the studies included in the meta-analysis, which might limit the interpretability of the pooled results.

Additionally, when they evaluated the OR as a measure of association, most of the results presented a non-significant association with a high level of heterogeneity, which might affect the overall effect. Although heterogeneity is common in systematic reviews and meta-analyses due to variations in study design, population, and methodologies, the authors did not provide a detailed exploration or explanation for the source of heterogeneity, which is a limitation.

In summary, the methods used for the statistical combination of results in this meta-analysis seem generally appropriate, but the high level of heterogeneity and the limited discussion around it suggests there might be considerable uncertainty around the pooled results. Therefore, the assessment would be “Mostly Yes” with the above caveats.

Domain 12: If meta-analysis was performed, did the review authors assess the potential impact of RoB in individual studies on the results of the meta-analysis or other evidence synthesis?

Based on the provided text, the authors of the systematic review conducted a meta-analysis, but it’s not explicitly mentioned that they assessed the potential impact of risk of bias in individual studies on the results of the meta-analysis. They described their methods for conducting the meta-analysis, including the statistical tests and approaches they used, and the process of article selection. However, the text doesn’t mention methods specifically addressing the risk of bias, such as sensitivity analyses excluding studies at high risk of bias or subgroup analyses based on risk of bias.

So, the assessment for this domain is ‘No’. Elements that are missing from the description include:

An explicit discussion of how the authors assessed the risk of bias in the individual studies included in the meta-analysis.

Any use of sensitivity or subgroup analyses, or other methods, to evaluate the impact of potential bias in individual studies on the overall results of the meta-analysis.

Despite this, it’s important to note that the authors have considered and quantified the heterogeneity among studies using I2 statistics, which is a measure of the total variation across studies due to heterogeneity rather than chance. But this only addresses heterogeneity and does not directly tackle the risk of bias. For a thorough assessment of the potential impact of risk of bias in individual studies, additional analyses or steps specifically designed to address this issue are needed.

Domain 13: Did the review authors account for RoB in individual studies when interpreting/ discussing the results of the review?

Based on the information provided, the authors of this systematic review followed the PRISMA guidelines, suggesting a systematic and comprehensive approach to their literature search, selection of studies, and data analysis. However, it is not explicitly stated whether they accounted for the risk of bias in the individual studies when interpreting or discussing the results of the review.

PRISMA guidelines recommend the assessment of the risk of bias of included studies in a systematic review, and its impact on the review findings. However, in the provided text, there’s no explicit mention of this process. Specifically, there’s no clear indication that they evaluated study characteristics such as blinding, selection bias, or potential conflicts of interest within the original studies.

Moreover, while they acknowledged variability (heterogeneity) across study results and quantified it using I² statistics and Tau², it doesn’t seem that they fully discussed the implications of this heterogeneity on their results. The high degree of heterogeneity could indicate a high risk of bias across the studies, which might limit the generalizability of the results and may warrant a more cautious interpretation.

Therefore, based on the information provided, the assessment of this domain is ‘No’. The systematic review lacks explicit discussion of the risk of bias in individual studies and does not clearly outline how this risk might have influenced the interpretation or discussion of the review’s results. For a more thorough evaluation of bias and its potential impact on the review, the authors could have used tools like the Cochrane risk of bias tool or ROBINS-I for non-randomized studies.

Domain 14: Did the review authors provide a satisfactory explanation for, and discussion of, any heterogeneity observed in the results of the review?

From the given text, it appears that the authors did provide an explanation for heterogeneity observed in the results of the review. They have quantified the heterogeneity through the Tau2 test and I^2 statistics. The authors mention that the I^2 statistics indicate appreciable and significant variability among studies, with ranges also greater than 50%.

However, it is unclear whether the authors performed sensitivity analyses or meta-regression to explore the reasons for heterogeneity. Subgroup analyses were performed, as stated in the “Data analysis” section, but the authors didn’t provide details on the results or impact of these analyses, which would have helped to better understand the source of heterogeneity.

Therefore, the assessment of this domain is “No”. The authors do recognize and report on the heterogeneity in their results, but they do not provide a detailed discussion of its sources or attempt to investigate it with methods such as sensitivity analysis or meta-regression. Providing these would have offered a more robust understanding of the source of heterogeneity, which in turn would have allowed for better interpretation of the results.

Domain 15: If they performed quantitative synthesis did the review authors carry out an adequate investigation of publication bias (small study bias) and discuss its likely impact on the results of the review?

The authors of the systematic review do not appear to have performed an adequate investigation of publication bias, also known as small study bias. The methods section lacks information on any strategies used to detect or account for publication bias. Techniques such as funnel plots, tests for small study effects, or other appropriate methods are not mentioned in the text.

Publication bias is a critical aspect to address in a systematic review and meta-analysis as it can lead to an overestimation or underestimation of the true effect size. It occurs when the results of studies influence the decision whether to publish or otherwise distribute them. Smaller studies, which often find no effect, are less likely to be published, leading to an overrepresentation of studies with significant findings.

The lack of a formal evaluation and discussion of publication bias can limit the validity and generalizability of the meta-analysis findings, as the included studies may not represent all relevant studies on the topic, and may bias the results towards positive or statistically significant outcomes.

Therefore, while the authors seem to have followed a rigorous process in other aspects of their systematic review and meta-analysis, the lack of an adequate investigation of publication bias is a significant limitation of their study.

Domain 16: Did the review authors report any potential sources of conflict of interest, including any funding they received for conducting the review? Based on the text provided, the authors reported no conflicts of interest. Therefore, the assessment for this domain is “Yes”.