Forum Discussion

RLRE's avatar
RLRE
Occasional Contributor
2 years ago

How can I get the whole image of a canvas object?

Hello all,

In the web app I am testing, there is a <canvas/> object that displays as image only. The app only displays part of the image because this image is almost always way too big to display. The user can scroll in all directions to view the other regions of the image.
I want to use TestComplete to save the entire image so that I can make image-level comparisons.
Using pageObj.PagePicture(), as expected, is not the solution because it only shows the page of the app.
Calling canvasObj.Picture(0,0,-1,-1,false) returns an image of the correct size, but of what is currently there on the screen (Including black areas for the part of the image that is outside the PC monitor).

How can I get the whole image of a canvas object at once?

Notes / More info:
1) If possible, I would like to avoid scrolling in the automation and shooting multiple images that I then have to 'glue' together. For similar reasons, I would like to avoid using region checkpoints for image parts if possible.
2) Development will give me a way to read the data I want the application to show in the image from the page's javascript via get methods. However, this is an additional nicety. I really need to check the displayed image, though. (!) It is not enough that the application sends the correct data to the client: The application must also display this data appropriately on the image. Testing the image directly is an important requirement of the client. Therefore I need to read the whole image of the canvas.

Thank you very much for your time!

  • RLRE's avatar
    RLRE
    2 years ago

     

    First of all: Thanks to rraghvani for the hints that put me on the right track. 

    Here is commented code with solution:

    // File names
    const fileNameRoot = "C:\\Temp\\Canvas";
    const fileNameHtml = fileNameRoot + ".html";
    const fileNamePng = fileNameRoot + ".png";
    const fileNameJpeg = fileNameRoot + ".jpeg";
    const fileNameWebp = fileNameRoot + ".webp";
    // get the page objekt - adapt parameters as needed...
    var pageObj = Sys.Browser(Project.Variables.Browser).Page(Project.Variables.HomePage + "*");
    // get the canvas object - we use here a xpath expression.
    var xPath = "//canvas[contains(@class, 'shapes')]"; // adapt as needed 
    var canvasObj = pageObj.FindChildByXPath(xPath);
    // following fails to get hidden canvas picture partitions
    // var pictureCanvas = canvasObj.Picture(0,0,-1,-1,false);
    /////////
    // Get canvas picture using canva native method toDataURL(). See: 
    // https://developer.mozilla.org/en-US/docs/Web/API/HTMLCanvasElement/toDataURL
    // The return value of toDataURL() is ready to embed in a html page and 
    // contains 'data&colon;image/png;base64' image at the begining
    var imagePNGAsBase64WithPrefix = canvasObj.toDataURL("image/png", 1.0); // 1.0 not needed
    // JPEG Contains 'data&colon;image/jpeg;base64,' and so on.
    var imageJPGAsBase64WithPrefix = canvasObj.toDataURL("image/jpeg", 1.0);  // 1.0 == no compression
    var imageWEBPsBase64WithPrefix = canvasObj.toDataURL("image/webp", 1.0);
    // we need to dropp prefixes for image file creation 
    const prefixHTML_imagePng = "data&colon;image/png;base64,"; 
    const prefixHTML_imageJpg = "data&colon;image/jpeg;base64,"; 
    const prefixHTML_imageWbp = "data&colon;image/webp;base64,"; 
    // dropp the prefixes
    var imagePNGAsBase64NoPrefix = aqString.Replace(imagePNGAsBase64WithPrefix, prefixHTML_imagePng, "", false);
    var imageJPGAsBase64NoPrefix = aqString.Replace(imageJPGAsBase64WithPrefix, prefixHTML_imageJpg, "", false);
    var imageWEBPsBase64NoPrefix = aqString.Replace(imageWEBPsBase64WithPrefix, prefixHTML_imageWbp, "", false); 
    // Create html file showing the images 
    var oFile = aqFile.OpenTextFile(fileNameHtml, aqFile.faWrite, aqFile.ctANSI, true);
    oFile.Write("<!DOCTYPE html><html><head><title>Canvas picture as PNG, JPEG and WebPage-Image</title></head><body>");
    oFile.Write("<img style='width:100%; height:100%;' src='" + imagePNGAsBase64WithPrefix + "'/>");
    oFile.Write("<img style='width:100%; height:100%;' src='" + imageJPGAsBase64WithPrefix + "'/>");
    oFile.Write("<img style='width:100%; height:100%;' src='" + imageWEBPsBase64WithPrefix + "'/>");
    oFile.Write("</body></html>");
    oFile.Close();
    // Attach the html file to log
    Log.Link(fileNameHtml, "HTML-File containing image from canvas only.");
    //////////////////////////////
    // Image file creation
    // Create png-File directly 
    var ps = dotNET.System_Management_Automation.PowerShell.Create();
    var psCmd = "[IO.File]::WriteAllBytes('" + fileNamePng + "',[Convert]::FromBase64String('" + imagePNGAsBase64NoPrefix + "'))";
    // log the actual command for fun.
    Log.Message("Power-Shell command", psCmd);
    ps.AddScript(psCmd); // 'type' it to PowerShell
    var results = ps.Invoke(); // Run.
    // Attach picture file to Log  
    Log.File(fileNamePng, "Canvas picture as PNG.");
    // Create JPEG-File directly
    psCmd = "[IO.File]::WriteAllBytes('" + fileNameJpeg + "',[Convert]::FromBase64String('" + imageJPGAsBase64NoPrefix + "'))";
    ps.AddScript(psCmd);
    results = ps.Invoke();
    Log.File(fileNameJpeg, "Canvas picture as JPEG");
    // Create WEBP-File directly
    psCmd = "[IO.File]::WriteAllBytes('" + fileNameWebp + "',[Convert]::FromBase64String('" + imageWEBPsBase64NoPrefix + "'))";
    ps.AddScript(psCmd);
    results = ps.Invoke();
    Log.File(fileNameWebp, "Canvas picture as WEBP");
    ///////////////////////////////
    // Compare with file from Disk
    var pictActual = Utils.Picture;
    pictActual.LoadFromFile(fileNameJpeg);
    // pictExpected is the name of expected result file (created by first run, for example).
    var pictExpected = Utils.Picture;
    pictExpected.LoadFromFile(fileNameRoot + "_Expected.jpeg");
    // the comparisson
    var bRes = pictActual.Compare(pictExpected)
    if (bRes) {
      Log.Checkpoint("Canvas picture as expected!");
    } else {
      Log.Error("Canvas picture don't meet expected result");
    }
    

     A last remark on the files created: Perhaps we may use toBlob() method instead of toDataUrl() to avoid conversions. (I haven't try it yet.)

    Good luck!

  • rraghvani's avatar
    rraghvani
    Champion Level 3

    Here's a small example based on the following website,

     

    function Canvas()
    {
        // https://www.w3schools.com/html/tryit.asp?filename=tryhtml5_canvas_tut_path
        var browser = Sys.Browser("edge");
        var page = browser.Page("https://www.w3schools.com/html/tryit.asp?filename=tryhtml5_canvas_tut_path");
        var canvas = page.FindElement("#iframeResult").FindElement("#myCanvas")
    
        // Contains 'data&colon;image/png;base64' image.
        var image = canvas.toDataURL("image/png");
    
        // Create html file
        var oFile = aqFile.OpenTextFile("C:\\Temp\\Canvas.html", aqFile.faWrite, aqFile.ctANSI, true);
        oFile.Write("<!DOCTYPE html><html><head><title>Page Title</title></head><body>");
        oFile.Write("<img style='width:100%; height:100%;' src='" + image + "'/>");
        oFile.Write("</body></html>");
        oFile.Close();
    }

     

    which converts the canvas object into a base64 image, which is then output to a html file,

    Unfortunately, I don’t have a.Net license to convert the base64 image to an actual .png file.

  • rraghvani's avatar
    rraghvani
    Champion Level 3

    Using PowerShell, you can pass the image variable (from above code) into the parameter base64image

     

    [IO.File]::WriteAllBytes('C:\Temp\Canvas.png', [Convert]::FromBase64String(base64image))

     

     to create C:\Temp\Canvas.png image file.

    • RLRE's avatar
      RLRE
      Occasional Contributor

       

      First of all: Thanks to rraghvani for the hints that put me on the right track. 

      Here is commented code with solution:

      // File names
      const fileNameRoot = "C:\\Temp\\Canvas";
      const fileNameHtml = fileNameRoot + ".html";
      const fileNamePng = fileNameRoot + ".png";
      const fileNameJpeg = fileNameRoot + ".jpeg";
      const fileNameWebp = fileNameRoot + ".webp";
      // get the page objekt - adapt parameters as needed...
      var pageObj = Sys.Browser(Project.Variables.Browser).Page(Project.Variables.HomePage + "*");
      // get the canvas object - we use here a xpath expression.
      var xPath = "//canvas[contains(@class, 'shapes')]"; // adapt as needed 
      var canvasObj = pageObj.FindChildByXPath(xPath);
      // following fails to get hidden canvas picture partitions
      // var pictureCanvas = canvasObj.Picture(0,0,-1,-1,false);
      /////////
      // Get canvas picture using canva native method toDataURL(). See: 
      // https://developer.mozilla.org/en-US/docs/Web/API/HTMLCanvasElement/toDataURL
      // The return value of toDataURL() is ready to embed in a html page and 
      // contains 'data&colon;image/png;base64' image at the begining
      var imagePNGAsBase64WithPrefix = canvasObj.toDataURL("image/png", 1.0); // 1.0 not needed
      // JPEG Contains 'data&colon;image/jpeg;base64,' and so on.
      var imageJPGAsBase64WithPrefix = canvasObj.toDataURL("image/jpeg", 1.0);  // 1.0 == no compression
      var imageWEBPsBase64WithPrefix = canvasObj.toDataURL("image/webp", 1.0);
      // we need to dropp prefixes for image file creation 
      const prefixHTML_imagePng = "data&colon;image/png;base64,"; 
      const prefixHTML_imageJpg = "data&colon;image/jpeg;base64,"; 
      const prefixHTML_imageWbp = "data&colon;image/webp;base64,"; 
      // dropp the prefixes
      var imagePNGAsBase64NoPrefix = aqString.Replace(imagePNGAsBase64WithPrefix, prefixHTML_imagePng, "", false);
      var imageJPGAsBase64NoPrefix = aqString.Replace(imageJPGAsBase64WithPrefix, prefixHTML_imageJpg, "", false);
      var imageWEBPsBase64NoPrefix = aqString.Replace(imageWEBPsBase64WithPrefix, prefixHTML_imageWbp, "", false); 
      // Create html file showing the images 
      var oFile = aqFile.OpenTextFile(fileNameHtml, aqFile.faWrite, aqFile.ctANSI, true);
      oFile.Write("<!DOCTYPE html><html><head><title>Canvas picture as PNG, JPEG and WebPage-Image</title></head><body>");
      oFile.Write("<img style='width:100%; height:100%;' src='" + imagePNGAsBase64WithPrefix + "'/>");
      oFile.Write("<img style='width:100%; height:100%;' src='" + imageJPGAsBase64WithPrefix + "'/>");
      oFile.Write("<img style='width:100%; height:100%;' src='" + imageWEBPsBase64WithPrefix + "'/>");
      oFile.Write("</body></html>");
      oFile.Close();
      // Attach the html file to log
      Log.Link(fileNameHtml, "HTML-File containing image from canvas only.");
      //////////////////////////////
      // Image file creation
      // Create png-File directly 
      var ps = dotNET.System_Management_Automation.PowerShell.Create();
      var psCmd = "[IO.File]::WriteAllBytes('" + fileNamePng + "',[Convert]::FromBase64String('" + imagePNGAsBase64NoPrefix + "'))";
      // log the actual command for fun.
      Log.Message("Power-Shell command", psCmd);
      ps.AddScript(psCmd); // 'type' it to PowerShell
      var results = ps.Invoke(); // Run.
      // Attach picture file to Log  
      Log.File(fileNamePng, "Canvas picture as PNG.");
      // Create JPEG-File directly
      psCmd = "[IO.File]::WriteAllBytes('" + fileNameJpeg + "',[Convert]::FromBase64String('" + imageJPGAsBase64NoPrefix + "'))";
      ps.AddScript(psCmd);
      results = ps.Invoke();
      Log.File(fileNameJpeg, "Canvas picture as JPEG");
      // Create WEBP-File directly
      psCmd = "[IO.File]::WriteAllBytes('" + fileNameWebp + "',[Convert]::FromBase64String('" + imageWEBPsBase64NoPrefix + "'))";
      ps.AddScript(psCmd);
      results = ps.Invoke();
      Log.File(fileNameWebp, "Canvas picture as WEBP");
      ///////////////////////////////
      // Compare with file from Disk
      var pictActual = Utils.Picture;
      pictActual.LoadFromFile(fileNameJpeg);
      // pictExpected is the name of expected result file (created by first run, for example).
      var pictExpected = Utils.Picture;
      pictExpected.LoadFromFile(fileNameRoot + "_Expected.jpeg");
      // the comparisson
      var bRes = pictActual.Compare(pictExpected)
      if (bRes) {
        Log.Checkpoint("Canvas picture as expected!");
      } else {
        Log.Error("Canvas picture don't meet expected result");
      }
      

       A last remark on the files created: Perhaps we may use toBlob() method instead of toDataUrl() to avoid conversions. (I haven't try it yet.)

      Good luck!

      • RLRE's avatar
        RLRE
        Occasional Contributor

        Method someCanvas.toBlob() seems to be inaccessible in TestComplete, why ever.