We present MAJEPPA, a self-supervised framework to learn piano performance representations that span the full skill spectrum, from beginner practice sessions to virtuoso concert recordings. We curate the MAJEPPA dataset, comprising ~4,000 annotated recordings across six expertise levels and six recording contexts. We adapt a single pre-trained MIDI autoregressive model with a joint objective: next-token prediction learns score-conditioned performance generation at various skill levels, while InfoNCE and supervised contrastive losses align abstract score and performance representations in a joint embedding space. The proposed model both generates and understands performances in a unified framework. By introducing the EVPMR benchmark, a suite of downstream tasks spanning quality assessment, competition ranking, mistake and technique classification, we evaluate the learnt representations, demonstrating progress towards a real-world model for the piano performance space.

Q18. Write ONE line (in your own words) with the main take-home message from the paper.

The unified JEPA-based representation learning model provides versatile embedding space for understanding piano performance with the capability of generating the performance from score.

Q23. Main review and comments for the authors (to be completed before the discussion phase). Please summarize strengths and weaknesses of the paper. It is essential that you justify the reason for the overall evaluation score in detail. Keep in mind that belittling or sarcastic comments are not appropriate.

This paper proposes a unified representation learning framework for piano music and performance that covers a wide range of skill levels and performance contexts, while also providing the capability to generate performances from scores. The authors build a novel dataset that aligns scores and performances across six expert pianists and six recording contexts. The EVPMR benchmark validates the superiority of the proposed representation learning approach.

This paper is very well written, and I do not find any significant issues that need to be addressed.

The Related Work section introduces an assortment of relevant studies that are helpful for understanding recent progress in representation learning in the symbolic music domain.

The MAJEPPA dataset is well curated, covering diverse skill levels and performance contexts. I can imagine how much effort the authors invested in data collection and processing, particularly in pairing performance MIDI data with sheet music and condition labels.

The JEPA-based model is not entirely new, but the conditioning mechanism and loss functions are carefully designed to support both generative modeling and representation learning.

The evaluation setup is clearly described and comprehensive.

The demo website is particularly impressive. It provides additional result tables, presumably not cherry-picked, performance renderings, and interactive visualizations color-coded by performance conditions, mistake-detection plots, and other useful analyses.

The benchmark includes recent comparative models and clearly demonstrates both the outstanding performance of the proposed model and the necessity of the multiple loss functions.

The embedding visualizations in Figure 4 provide an intuitive understanding of performances in terms of skill levels and mistake types.

< Minor Comments>

• Line 127: EVAR is introduced without explanation or expansion of the acronym.
• I find the dataset name somewhat contradictory given its association with the notoriously difficult piano repertoire of Franz Liszt.
• The tile of the name could be reconsidered. The word "Morphing" needs some explanation to understand it and the word never appeared in the main body of the paper. The word "assessment" might give an impression that the application is limited to education.

Overall, this paper is well written and easy to read. The proposed model represents a significant step forward in piano music understanding. By combining learned representations with generative capabilities, the model could be extended to interactive applications that make performance analysis and piano education more intuitive and exploratory. I strongly recommend acceptance of this paper.

Q25. Meta-review and final comments for authors (to be completed after the discussion phase)

The paper presents MAJEPPA, a novel framework and carefully curated dataset designed to learn unified symbolic representations of piano music that span a wide spectrum of performance proficiency levels and recording contexts.

The reviewers unanimously praised the paper for its clarity, rigorous downstream evaluation on the proposed EVPMR benchmark, and a highly impressive, comprehensive interactive demo page. The adaptation of JEPA to symbolic music processing is recognized as a timely and successful contribution.

However, the reviewers noted a few areas that the authors must address in the camera-ready version. Specifically, the authors need to include an explicit ethical and copyright statement regarding the human annotators and the scraping performance data.

In addition, the final manuscript must clarify several technical ambiguities, including token packing parameters, the choice of the Transkun transcription model over Aria-AMT, how repeat-structure mismatches were handled during the DTW process, and the specific role of the generative loss in the ablation studies.

Q18. Write ONE line (in your own words) with the main take-home message from the paper.

The authors introduce a JEPA-based SSL to train an autoregressive model that learn MIDI representations that incorporate (and also differentiate between) score and performance characteristics. They also curate both an annotated dataset for this purpose, along with a set of evaluation tasks on which they assess their trained (and lightly ablated) model.

[NB: Yes I am aware this is TWO sentences; it almost could have been two papers]

Q23. Main review and comments for the authors. Please summarize strengths and weaknesses of the paper. It is essential that you justify the reason for the overall evaluation score in detail. Keep in mind that belittling or sarcastic comments are not appropriate.

This is a very nice paper that successfully incorporates a lot of work (adaptation of JEPA to MIDI, algorithmic details and implementation, dataset curation, and thorough evaluation). I suspect that this could (almost?) have been rewritten as two papers, should the authors have chosen to do so. Each of the multiple contributions is valuable and well-presented, and taken altogether I feel that this is a strong paper that I feel--equally strongly---merits acceptance.

I really like the high-level goal of a learning a unified representation for both generation and understanding of musical performances.

Applying the JEPA approach in this context is a timely idea, and it clearly must have taken a lot of important detail choices to get it to work. The demonstrations on the website are evidence that it is fairly successful on a generative level, and the performance on the EVPMR relative to well-chosen reasonable baselines is clearly effective! (Nice work!)

On one hand, the part of the paper that feels to me the least developed or refined, is the 6x6 categorization labels w.r.t. performer expertise and recording context. On the other hand, I feel it is absolutely sufficient in light of all the other contributions of the paper: the proposed annotations are (at least) a reasonable first-order approximation, and entirely appropriate in this context. I have no doubt that this dataset alone, with its current annotations, is already a valuable contribution to the community as well. I see this more as suggestive of next possible steps rather than requiring any modification in the present version.

There is one serious and glaring omission in my view, and that is the complete lack of discussion of ethics. This does not take any space from the page limit, and so there is no reason not to include it. Just to mention a few obvious questions that come to mind: Did the authors of the training set (e.g. youtube posters) consent in principle to this usage of their work? E.g. if a music teacher has posted educational videos in order to build a following and subscriber base, is this paper training a model on their data, so as to work towards an automated “skill-building” system that will replace them? What about the generative aspects of the system, training on “concerts”, e.g. “commercial recordings”, by professional pianists with the ability to subsequently condition the generation on ‘virtuoso’ pianists? Another ethical question perhaps worth addressing for other reasons is that of training on children’s recordings taken from the internet.

Small question: I think that in one of the demo’s on the website, I recall hearing a generated “practise” example where a certain section was getting repeated multiple times before proceeding on with the piece. This seems legitimate in terms of how practising sounds, but it wasn’t clear to me how such examples (presumably also found in the training set) get processed (e.g. as described in Sec 3.2, since DTW usually still requires a “monotonic” correspondence as opposed to looping/repetition of subsequences). And then does this later get encoded as a performance characteristic? (Or maybe I just mis-remembered the excerpt; tbh I didn’t dig up the score to verify it…)

Ideally, limitations would be discussed as well, but I recognize that 6 pages is very short, and clearly the authors made significant efforts to choose the material that needed to be included in order to fit their work into this already-compressed format, so I think discussion of limitations might simply be unreasonable to include. still, there are significant potential limitations both to the set of classifications (i.e. both w.r.t. recording context and w.r.t. expertise levels), and to the interpretation of some of the evaluation tasks, although the authors appropriately do not over-claim about the significance of the eval suite.

BTW, I appreciate the name-reference to the Liszt transcendental étude, as well as the emoji-sized horowitz in Fig 3. : )

Q18. Write ONE line (in your own words) with the main take-home message from the paper.

JEPA can be incorporated into symbolic piano music domain along with the generative loss for a joint performance understanding and generation capabilties.

Q23. Main review and comments for the authors. Please summarize strengths and weaknesses of the paper. It is essential that you justify the reason for the overall evaluation score in detail. Keep in mind that belittling or sarcastic comments are not appropriate.

The paper proposes MAJEPPA, a joint MIDI piano performance understanding and generation model that combines JEPA self-supervised objective with the next token prediction. Solo piano recordings from YouTube are transcribed into MIDI, resulting in a new dataset spanning multiples levels of performance proficiency and recording conditions. MAJEPPA shows strong performance on downstream symbolic music understanding tasks, such as performance ranking, pianist classification, and technique detection.

Strengths:

1. Excellent presentation: from the clear writing of the main text to illustrative examples and impressive interactive demo page.

2. The dataset constitutes a valuable contribution to MIR, as it differentiates between six degrees of piano proficiency, from beginner to virtuoso pianists. The recordings are manually annotated by humans, which explains relatively moderate dataset size (4,000 recordings). This has a potential to be a great resource for music education and analysis.

3. To my knowledge, the JEPA framework has not been applied before in the context of symbolic piano music understanding. While the method itself is not novel, the paper's adaptation of LLM-JEPA to MIR is novel.

4. MAJEPPA method outperforms recently proposed SSL models, such as Aria, CLaMP3, Moonbeam, and MidiBERT on several downstream tasks with a considerable margin.

Weaknesses:

1. The paper admits that experiments related to performance generation are too limited. While this task is interesting and generation results on the demo page sound promising and seem to differentiate between performance levels, they are not impressive. The ablation study does not explain how the generative loss affects the model performance. Does it help the model to better understand music? In contrast, does the JEPA framework lead to better music generation?

2. The paper does not discuss how the absence of pedaling tokens in Aria affects the performance. Moreover, the main text also does not discuss the limitations of Transkun audio-to-MIDI transcription.

Questions:

1. What is $k$ in Section 4.1 (Encoder)? I think the explanation of token packing is rather sparse and confusing.

2. Some of the performances sound rather strange in the 3D Main MAJEPPA embedding space (demo page). For example, the performance of Liszt's 12 "Etudes d'exécution transcendante, S.139 - No.10 Allegro agitato molto", which is labeled as "virtuoso" (seg000), sounds like a MIDI deadpan audio and played in an incredibly slow tempo. I suspect there might be some bugs either in demo or dataset. Moreover, there is a confusing heterogeneity in recordings, i.e., some of the performances sound like real audio recordings whereas others sound like MIDI transcriptions.

Suggestions:

1. Consider adding the "hours" column to Table 1, as it is not clear what is the average duration of recordings.

Overall, great contribution and a contender for the best paper award.

Q18. Write ONE line (in your own words) with the main take-home message from the paper.

A novel dataset and a framework for the multi-dimensional analysis of piano performances.

Q23. Main review and comments for the authors. Please summarize strengths and weaknesses of the paper. It is essential that you justify the reason for the overall evaluation score in detail. Keep in mind that belittling or sarcastic comments are not appropriate.

The paper introduces MAJEPPA, a framework and a curated dataset for learning unified representations of piano performances. The work is well motivated by an existing research gap: no annotated dataset and embedding models that encode a spectrum of different performance levels and recording contexts.

The application of a Joint-Embedding Predictive Architecture (JEPA) is well suited to generative/analytical piano performance modelling. The results of the downstream tasks on the developed EVPMR benchmark are highly convincing. The demo page, model and embedding playgrounds are excellent!

The weaknesses, concerns and questions:

• Inclusion of Yiruma and maybe other works raises potential copyright issues. Currently, there is no ethical statement in the paper. Also, human subjects were involved in the annotation process. Any potential issues with the data should be stated directly in the paper.
• There is no mention of how many musical pieces these 3,979 recordings comprise.
• Section 3.1: The paper mentions manual annotation by conservatoire students. Was each annotation performed by a single individual, or were multiple annotators used? Furthermore, what measures were taken to ensure inter-annotator reliability?
• Section 3.1: How were the score files chosen from MuseScore? Popular compositions have multiple editions and variations with different repeat structures, for example.
• Section 3.2: How were these discrepancies between score and performance repeat structures considered during the DTW alignment process? Specifically, how are missing/inserted segments handled in the annotated data?
• Section 4.3: The methodology states that the pre-trained Aria weights are frozen and LoRA is applied (L362-364). From the text, it is unclear what is meant by the plural form of the word "adapters".
• The embedding model works with symbolic music, not audio, and audio is only presented as one of the modalities of the MAJEPPA dataset. It is likely that acoustic information could provide richer embeddings for analysing performances across the presented expertise levels.
• Given that the experiments rely on audio-to-MIDI transcription using the Transkun model, were the transcriptions checked for any errors, noise and unrealistic pedal predictions. Since Aria models were trained on data produced by Aria-AMT, why was this model not considered?

Minor writing and presentation issues:

• Inconsistent use of hyphens: "encoder–decoder" (L102), "score–performance" (L119) but "composition-level" (L122), "score-informed" (L126).
• L170: extra bracket in "Progress)"
• The second sentence of Section 4.1 is difficult to understand on the first read.
• ArXiv citations should be updated to include the conference/journal references (Journal of Creative Music Systems for MidiBERT-Piano, ICLR for LLM-JEPA).

No other issues to mention. Hopefully, the MAJEPPA dataset and the model will be released with the final publication, as these will make a valuable contribution to the research community.

We thank the reviewers for their valuable feedback. Below we summarise the changes made to the submission.

Section 2: Added two additional relevant papers (R1).

Section 3: Added the number of distinct musical pieces/movements per recording (R3). Added a total-hours column to Table 1 (R2). Clarified the annotation process, including the number of annotators per recording (R3). Clarified the MuseScore edition selection (R3). Added an explanation of how subsequence DTW handles omitted and repeated sections (R1, R3). Added justification for the choice of the Transkun transcription model (R2, R3).

Section 4: Updated Figure 3 for clearer presentation of k (R2). Clarified the LoRA setup description (R3).

We added an ethics statement, and made minor writing and presentation improvements throughout.