Appraisal Systems Performance

The performance of an appraisal tool’s ability to predict NFT prices is evaluated via the following method:
  1. 1.
    A finite set of contemporary transaction pricing data is algorithmically selected for the desired NFT collection.
  2. 2.
    The model is called upon to predict the price of each NFT within the selected dataset.
  3. 3.
    The error between the price predicted by the model and the real traded price is evaluated for each NFT within the selected set.
  4. 4.
    Then, median absolute percent error, 95th percentile absolute percent error, and maximum absolute percent error, as well as the geometric mean of the three metrics (N Score), are calculated from the dataset of errors.
  5. 5.
    The model that evaluates with the lowest N Score is deemed the most accurate (and best) pricing model for the dataset.

Median Absolute Percent Error (MAPE)

MAPE is calculated by taking the absolute value of the aforementioned dataset of errors and evaluating the median. Below, we compare the MAPE of individual industry appraisal tools versus Spice’s aggregator over time.
Fig. 6: Evaluated MAPEs over time across NFT appraisal tools for BAYC collection.
Fig. 7: Evaluated MAPEs over time across NFT appraisal tools for CryptoPunks collection.
Fig. 8: Evaluated MAPEs over time across NFT appraisal tools for Meebits collection.
Fig. 9: Evaluated MAPEs over time across NFT appraisal tools for Doodles collection.
Fig. 10: Evaluated MAPEs over time across NFT appraisal tools for Otherdeed collection.

Normal Score (N Score)

N Score is calculated by taking calculated MAPE, 95APE, MaxAPE values for the aforementioned dataset of errors and evaluating the geometric mean. Below, we compare the N Score of individual industry appraisal tools versus Spice’s aggregator over time.
Fig. 11: Evaluated N Scores over time across NFT appraisal tools for BAYC collection.
Fig. 12: Evaluated N Scores over time across NFT appraisal tools for CryptoPunks collection.
Fig. 13: Evaluated N Scores over time across NFT appraisal tools for Meebits collection.
Fig. 14: Evaluated N Scores over time across NFT appraisal tools for Doodles collection.
Fig. 15: Evaluated N Scores over time across NFT appraisal tools for Otherdeed collection.