Built upon a multi-agent architecture comprising a leader agent, a kernel agent, a research agent and an enhancement agent (Fig. 1a and Supplementary Note 1), CopilotJ integrates task comprehension, software control, domain knowledge retrieval and advanced computation to support complex analysis workflows. Users specify desired analysis tasks through natural-language interaction, after which CopilotJ formulates analysis strategies, generates executable instructions, progressively completes the requested tasks and compiles interpretable reports upon request (Supplementary Figs. 2 and 3). Throughout this process, CopilotJ communicates with ImageJ via a dedicated bidirectional plug-in, the CopilotJ Bridge, to query the environment state and trigger execution. Crucially, once an analytical pipeline is validated on a representative instance, it can be further saved as a reusable workflow object and applied to an entire directory for batch processing.We showcase CopilotJ across a range of tasks spanning foundational operations, core computational imaging analyses, domain-specific quantification and comprehensive multistage pipelines (Fig. 1b, Supplementary Fig. 4 and Supplementary Videos 1–12). We further evaluated CopilotJ’s quantitative robustness using synthetic microscopic image datasets8. For these images with well-defined ground truth, CopilotJ consistently achieved accurate cell counting results across datasets with varying morphological characteristics (Supplementary Fig. 5 and Supplementary Note 1). Alongside task diversity and quantitative accuracy, CopilotJ also exhibited strong stability and low sensitivity to the choice of underlying LLM. Across a range of proprietary and open-source models, it achieved an 84–93% task-completion stability rate when each of the tasks was repeated 15 times (Supplementary Fig. 6 and Supplementary Note 1).Access Nature and 54 other Nature Portfolio journalsGet Nature+, our best-value online-access subscription34.99 € / 30 days cancel any timeLearn moreSubscribe to this journalReceive 12 print issues and online access269.00 € per year only 22.42 € per issueLearn moreBuy this articlePurchase on SpringerLinkInstant access to the full article PDF.39,95 €Prices may be subject to local taxes which are calculated during checkoutData availabilityThe data used in this study are either publicly available from their original sources or accessible via https://doi.org/10.5281/zenodo.20413892.Code availabilityCopilotJ is open source and released under the Apache-2.0 license. The complete source code is publicly available on GitHub at https://github.com/neurogeom/copilotj. In addition, CopilotJ Bridge can be downloaded through the ImageJ Updater as an ImageJ plug-in. A web-based chat interface is available at https://copilotj.chat.ReferencesSchneider, C. A., Rasband, W. S. & Eliceiri, K. W. Nat. Methods 9, 671–675 (2012).Article CAS PubMed PubMed Central Google Scholar Schindelin, J. et al. Nat. Methods 9, 676–682 (2012).Article CAS PubMed PubMed Central Google Scholar Rueden, C. T. et al. Nat. Methods 19, 1326–1327 (2022).Article CAS PubMed PubMed Central Google Scholar Gómez-de-Mariscal, E. et al. Nat. Methods 18, 1192–1195 (2021).Article PubMed Google Scholar Bran, A. M. et al. Nat. Mach. Intell. 6, 525–535 (2024).Article Google Scholar Lei, W., Fuster-Barceló, C., Reder, G., Muñoz-Barrutia, A. & Ouyang, W. Nat. Methods 21, 1368–1370 (2024).Article CAS PubMed Google Scholar Royer, L. A. Nat. Methods 21, 1371–1373 (2024).Article CAS PubMed Google Scholar Ljosa, V., Sokolnicki, K. L. & Carpenter, A. E. Nat. Methods 9, 637–637 (2012).Article CAS PubMed PubMed Central Google Scholar Xiao, C. et al. Nat. Mach. Intell. 7, 1823–1833 (2025).Article Google Scholar Rates, A., Passmore, J. B., Norlin, N. & Kapitein, L. C. Npj Imaging 4, 14 (2026).Article PubMed PubMed Central Google Scholar Download referencesAcknowledgementsWe thank C. Rueden for software testing, bug reporting, code contributions and valuable feedback during the review process. We thank C. Hui for advice and discussions throughout the project and thank X. Liu for participation in early-stage development. We also thank X. Teng, L. Liu, J. He and X. Tan for the support of discussion, testing and feedback.FundingThis work was supported by the National Natural Science Foundation of China under grant number 32571281 and 92570202.Author informationAuthor notesThese authors contributed equally: Zhaoqi Lu, Zexin Yuan.Authors and AffiliationsGuangdong Institute of Intelligence Science and Technology, Hengqin, ChinaZhaoqi Lu (路沼奇), Zexin Yuan (元泽欣), Yan Chen (陈琰), Zihao Chen (陈子浩), Zhihai Qiu (丘志海) & Yimin Wang (王宜敏)Department of Biomedical Engineering, The Hong Kong Polytechnic University, Hong Kong SAR, ChinaZhaoqi Lu (路沼奇), Zihao Chen (陈子浩) & Lei Sun (孫雷)Key Laboratory of Cell Proliferation and Regulation Biology of Ministry of Education, The Center for Biological Science and Technology, Zhuhai-Macao Biotechnology Joint Laboratory, Beijing Normal University at Zhuhai, Zhuhai, ChinaZexin Yuan (元泽欣) & Haiyun Ren (任海云)AuthorsZhaoqi Lu (路沼奇)View author publicationsSearch author on:PubMed Google ScholarZexin Yuan (元泽欣)View author publicationsSearch author on:PubMed Google ScholarYan Chen (陈琰)View author publicationsSearch author on:PubMed Google ScholarZihao Chen (陈子浩)View author publicationsSearch author on:PubMed Google ScholarZhihai Qiu (丘志海)View author publicationsSearch author on:PubMed Google ScholarHaiyun Ren (任海云)View author publicationsSearch author on:PubMed Google ScholarLei Sun (孫雷)View author publicationsSearch author on:PubMed Google ScholarYimin Wang (王宜敏)View author publicationsSearch author on:PubMed Google ScholarContributionsY.W. conceived and managed the study. Z.L. and Z.Y. developed the software. Y.W., Z.L., Z.Y., Y.C. and Z.C. designed the experiments. L.S., H.R. and Z.Q. supported the project and offered resources to accomplish this research. Y.W., Z.L. and Z.Y. wrote the manuscript with assistance of all authors, who reviewed and revised the manuscript.Corresponding authorsCorrespondence to Haiyun Ren (任海云), Lei Sun (孫雷) or Yimin Wang (王宜敏).Ethics declarationsCompeting interestsThe authors declare no competing interests.Peer reviewPeer review informationNature Methods thanks Saul Kato, who co-reviewed with Raymond Dunn; Constantin Pape, who co-reviewed with Anwai Archit; and Curtis Rueden for their contribution to the peer review of this work. Primary Handling Editor: Fariha Rahman, in collaboration with the Nature Methods team.Supplementary informationSupplementary Information (download PDF )Supplementary Notes 1–3, Figs. 1–11, Tables 1–4 and screenshots of Videos 1–17Supplementary Video 1 (download MP4 )CopilotJ performs basic manipulation. CopilotJ executes a multi-step image-processing workflow, including duplication, rotation, contrast adjustment, blurring, Otsu thresholding, and edge detection. All intermediate results are automatically saved and assembled into a labeled grid figure, making the analysis procedure clear and easy to inspect.Supplementary Video 2 (download MP4 )CopilotJ performs video generation. CopilotJ generates a 3D visualization video from neuron image data; when visibility is insufficient, background noise reduction and contrast enhancement are automatically applied prior to video generation.Supplementary Video 3 (download MP4 )CopilotJ performs batch image post-processing. CopilotJ processes a folder of Nissl-stained images by automatically adjusting contrast, adding a standardized scale bar (1000 μm), and exporting the results with sequential filenames.Supplementary Video 4 (download MP4 )CopilotJ performs image deconvolution. CopilotJ performs deconvolution on a Drosophila image stack using a provided point spread function (PSF) and exports the restored volume for visualization and analysis.Supplementary Video 5 (download MP4 )CopilotJ performs human-in- the-loop segmentation. CopilotJ first surveys and recommends commonly used segmentation strategies for the given image, then executes a threshold-based pipeline selected by the user. During execution, the threshold is manually adjusted by the user rather than fixed automatically, and CopilotJ applies the selected value to generate the final segmentation output, enabling transparent human-in-the-loop control over the segmentation process.Supplementary Video 6 (download MP4 )CopilotJ performs object tracking in time-lapse images. CopilotJ detects and tracks all objects in a time-lapse movie and automatically outputs quantitative motion statistics together with trajectory visualizations.Supplementary Video 7 (download MP4 )CopilotJ performs colocalization analysis. CopilotJ performs red- green channel colocalization analysis on the input image and saves the resulting quantitative metrics and corresponding output files, enabling standardized and reproducible colocalization measurements.Supplementary Video 8 (download MP4 )CopilotJ performs comet assay analysis. CopilotJ automatically segments comet assay heads and tails, quantifies tail length and the percentage of DNA in the tail, and exports both the numerical measurements and corresponding overlay images.Supplementary Video 9 (download MP4 )CopilotJ performs Gel electrophoresis analysis. CopilotJ quantifies lane signal across six lanes within a user-selected rectangular ROI in an SDS-PAGE image and exports the per-lane measurements. The results are then automatically summarized as a bar chart, providing an interpretable visualization of lane-to-lane differences.Supplementary Video 10 (download MP4 )CopilotJ performs cell scratch assay quantification. CopilotJ quantifies wound area in cell scratch assay images at t0 and t24 using the maximum-gap method and calculates the wound-closure percentage between the two time points.Supplementary Video 11 (download MP4 )CopilotJ performs automated tissue morphology analysis and workflow management. CopilotJ performs automated glomeruli segmentation and tissue morphology quantification, generating segmentation masks, result tables, and analysis plots, including a correlation matrix computed from area, circularity, and solidity. The analysis dialog is then saved as a workflow, which can be listed, retrieved, and re- applied to all images within the same directory, with CopilotJ automatically executing the workflow in batch and organizing outputs into separate result folders for each image.Supplementary Video 12 (download MP4 )CopilotJ performs automated calcium imaging analysis. CopilotJ processes a two-photon calcium imaging time-lapse stack by performing full-movie motion correction, automatically segmenting all neurons, assigning unique ROI IDs, and extracting centroid coordinates. It then extracts ROI and surrounding- background fluorescence, computes background-corrected traces (F – 0.7·background), aligns responses to ultrasound stimulation frames using a 5 s pre- stimulus baseline and a 25 s post- stimulus window, and produces both ΔF/F0 and baseline-derived z- scored signals per neuron and per stimulation. Responses are classified as activated, inhibited, or non-responsive, with "true activation" requiring z ≥ 2 and ΔF/F0 > 2 within 3 s after stimulation; all outputs (motion- corrected TIFFs, ROI masks with IDs/centroids, corrected traces, per-neuron plots, aligned traces, labels, and summary tables including counts and success.Supplementary Video 13 (download MP4 )Comparing CopilotJ with the original interactively-used ImageJ. To evaluate CopilotJ’s real-world performance relative to human users, we conducted side- by-side comparisons across representative bioimage analysis tasks. Up: CopilotJ executes the same task automatically based on a single natural-language prompt. Bottom: Human participants manually performed the sequence of ImageJ operations (duplicate, rotate 90 degrees, adjust contrast, blur, Otsu thresholding and find edges), saving each intermediate result and assembling a final labeled grid figure.Supplementary Video 14 (download MP4 )CopilotJ performs image super- resolution. CopilotJ applies a deep-learning-based super- resolution model implemented with BiaPy by loading the specified pretrained checkpoint and enhancing image clarity and structural detail, producing high- resolution outputs for improved visualization and downstream analysis.Supplementary Video 15 (download MP4 )CopilotJ performs cell classification. CopilotJ runs inference using a pretrained classification model to identify cell types in the input images, producing predicted labels and structured classification outputs.Supplementary Video 16 (download MP4 )CopilotJ performs cell profiling with Cellpose. CopilotJ performs cell profiling by combining deep- learning-based segmentation using Cellpose with classical image- processing methods, and displays the resulting segmentation and quantitative profiling outputs for comparative analysis and interpretation.Supplementary Video 17 (download MP4 )Installation and configuration of CopilotJ. The CopilotJ plug-in can be installed and updated through the Fiji Updater, with core server deployment and startup performed directly from the plug-in. In addition, an onboarding wizard guides users through the environment configuration process.Rights and permissionsReprints and permissionsAbout this article