国际评价科学与工程研讨会接收摘要投稿！

主题包括但不限于以下内容：

1. Evaluation theory and methodology

• Formal specification of evaluation requirements

• Development of evaluation models

• Design and implementation of evaluation systems

• Analysis of evaluation risk

• Cost modeling for evaluations

• Accuracy modeling for evaluations

• Evaluation traceability

• Identification and establishment of evaluation conditions

• Equivalent evaluation conditions

• Design of experiments

• Statistical analysis techniques for evaluations

• Methodologies and techniques for eliminating confounding factors in evaluations

• Analytical modeling techniques and validation of models

• Simulation and emulation-based modeling techniques and validation of models

• Development of methodologies, metrics, abstractions, and algorithms specifically tailored for evaluations

2. The engineering of evaluation

• Benchmark design and implementation

• Benchmark traceability

• Establishing the least equivalent evaluation conditions

• Index design, implementation

• Scale design, implementation

• Evaluation standard design and implementations

• Evaluation and benchmark practice

• Tools for evaluations

• Real-world evaluation systems

• Testbed

3. Data set

• Explicit or implicit problem definition deduced from the data set

• Detailed descriptions of research or industry datasets, including the methods used to collect the data and technical analyses supporting the quality of the measurements

• Analyses or meta-analyses of existing data

• Systems, technologies, and techniques that advance data sharing and reuse to support reproducible research

• Tools that generate large-scale data while preserving their original characteristics

• Evaluating the rigor and quality of the experiments used to generate the data and the completeness of the data description

4. Benchmarking

• Summary and review of state-of-the-art and state-of-the-practice

• Searching and summarizing industry best practice

• Evaluation and optimization of industry practice

• Retrospective of industry practice

• Characterizing and optimizing real-world applications and systems

• Evaluations of state-of-the-art solutions in the real-world setting

5. Measurement and testing

• Workload characterization

• Instrumentation, sampling, tracing, and profiling of large-scale, real-world applications and systems

• Collection and analysis of measurement and testing data that yield new insights

• Measurement and testing-based modeling (e.g., workloads, scaling behavior, and assessment of performance bottlenecks)

• Methods and tools to monitor and visualize measurement and testing data

• Systems and algorithms that build on measurement and testing-based findings

• Reappraisal of previous empirical measurements and measurement-based conclusions

• Reappraisal of previous empirical testing and testing-based conclusions