siangyu.

research

my previous work focused on computer vision with multimodal large language models. i’m now shifting my research toward AI safety.

  1. TGPED: Task-Generalized Procedural Error Detection and Explanation using Large Language Models with Procedural Grounding

    TGPED framework: PGTS segments an input video using a procedural reference, then a shared detector identifies errors, explains them, and localizes omitted steps.

    Siangyu Yang, S-Wei Chen, You-Zhe Xie, Ho-En Tsai, Yu-Chi Lai, Kai-En Peng, YouChen Lai, Yu-Lun Liu, Chun-Yi Lee, Shih-Po Lee, Shao-Yuan Lo

    Under review at ICLR 2027

    openreview

    A task-generalized approach to detecting and explaining errors in procedural videos, combining training-free temporal segmentation with synthetic error descriptions for model training.

  2. 多模態大型語言模型於專業攝影評論推理與自動評分之評測研究

    MLLM 與攝影專家評分的 Pearson 相關係數比較，涵蓋曝光數值建議一致性、數值建議精確性與額外技術建議。

    Yi-Lin Tsai, Siangyu Yang, I-Sheng Fang, Jun-Cheng Chen

    Accepted at TAAI 2025 (Poster)

    我們評估多模態大型語言模型是否能比較專業攝影評論，並理解光線、構圖與對焦等攝影概念。結果顯示，模型與專家評分的相關係數皆低於 0.4，且在交換評論順序後，判斷的一致性有限。不同前沿模型之間也未呈現顯著差異，顯示目前模型在專業攝影推理上仍有進步空間。