Friday, March 02, 2001 ..:: Home ::.. Register  Login
   TechTidBits (Blog)  

Playing with Graphics/GDI+ to screen scrape the Habs current web page

Jun 16

Written by:
Tuesday, June 16, 2009 10:21 PM  RssIcon

I'm working on a Montreal Canadiens screen saver and thought grabbing a copy of the current web page would be cool, but how to do it and also how can you add "stuff" to it?  Read on for some cool graphic details (haha).

I've been playing around with graphics a bit and trying to make a screen saver for my favourite team, the Montreal Canadiens (aka the Habs).  I thought a fantastic idea would be to have the Habs current webpage as the main body for a screen saver!  Why not?  It's constantly changing and being updated, it's dynamic and will give me information I'm usually interested in (unless of course it's the off-season and they're all GOLFING!!!!!!!!  GGGGGGRRRRRR).  But anyways, I digress, I have created a small solution to screen scrape the current Habs webpage and do some thing with it.  This solution has three projects, one which simply grabs the webpage and saves it to a temp file, and the next two have slight improvements to them with the last one using events better and also adding some text to the graphics.  I hope you download and play with the code.  After all, what else is there to do for us Habs fans until the next season starts?  haha

The first project simply downloads the current webpage, strips out the vertical scrollbar and then saves it to the temp directory.

First Sample Project

This sample is an example of using the WebBrowser object and the DocumentComleted event as well as stripping out the vertical scrollbar.  You'll notice, I'm NOT displaying the webpage using the WebBrowser.  Then why am I using it?  To "render" the html with graphics only, then I create a Bitmap object with the width equal to the webapge minus the vertical scroll bar's width (this effectively strips it out, no matter what the size is on the user's computer) then I save the bitmap the file system for save keeping and then set the picture boxes Image property to that newly created bitmap (the in memory one, not the one on the file system).

private void HabsBrowser_DocumentCompleted( object sender, WebBrowserDocumentCompletedEventArgs e )
  Status( "Finished downloading webpage" );

  Control control = habsBrowser;
  Rectangle webpageRectangle = habsBrowser.Document.Body.ScrollRectangle;

  webpageRectangle.Width += GetScrollbarWidth();

  habsBrowser.Bounds = webpageRectangle;
  Bitmap bitmap = new Bitmap( webpageRectangle.Width - GetScrollbarWidth(), webpageRectangle.Height );
  control.DrawToBitmap( bitmap, webpageRectangle );

  string habsFileName = "C:\\temp\\HabsRule." + ImageFormat.Png.ToString().ToUpper();
  bitmap.Save( habsFileName, ImageFormat.Png );
  StatusLabel.Text = "Saved PNG go see it " + habsFileName;
  WebPictureBox.Image = bitmap;

Then only thing magical about this piece of code is the GetScrollBarWidth();

    private int GetScrollbarWidth()
      return new VScrollBar().Width;

The magic here is, no matter who runs this, juuuuust the right amount will be trimmed off the webpage to effectively strip out the scrollbar.  The only way this would screw up was if your user was changing their border/scrollbar sizes at exactly the wrong time (after it's created the scrollbar to get the width and creating the new Bitmap with the subtracted width (which is HIGHLY unlikely and if it did, it would likely be only a one shot deal, so I think you're safe doing this).

The next sample project, shows how to intercept different pages that are loaded then figuring how which one is the one we want to start drawing our image on.

 Showing different events

The last project is the most interesting (if I say so myself haha).  GDI+ is stateless which means if you want to draw something, not only do you have to draw it once, but you have to remember it for the next time you want to draw it, which could very well likely in about 1/32 of a second!  Yup, if you resize your window, or if you move a window over top of your window, each one of those will generate LOTS'n'LOTS of Paint events!  Try it out, put a Console.WriteLine("Painting"); in your Paint event handler to see how often it's called, then move the window around, resize it, min/max the window.

If you find your images are disappearing on you, after you first draw them, chances are you are doing your drawing in a button Click event or something like that.  What I think you might want to do instead is have a local variable to keep what you want to display (like my private image habsWebPageImage;) then you will change that variable as well as use that variable to paint onto the control.

Here's what the end result is of the last project.

Adding graphics

In this example, I'm widening the picture by the width of new graphics, turned on it's edge with a gradient background and adding both my new cheer graphics to the webpage graphic and saving that to the file system.

Here's the magical DocumentCompleted method that does all the graphic magic.  Notice how there is VERY LITTLE hard coded.  Maybe the font size (80pts) , everything else is, and SHOULD be relative to something else.  That's the key piece here with successful graphics for WIndows IMHO!  Everything is relative to something else!  No matter what, you're inside of another control, even if you're in a maximimized window, you're still inside a window!  If you find yourself hard coding a height or width, try to ask yourself if there is any way, shape or form you can derive that value from something else?  In my case, I could use a % for the width of that cheer graphics, that would be a cool addition!

private void HabsBrowser_DocumentCompleted( object sender, WebBrowserDocumentCompletedEventArgs e )
      //need to do this cause when the WebBrowser "Navigates" to a webpage, it fires off several Navigating events
      //and thus has several DocumentCompleted firing as well, we only want to do this on the whole page
      //this is the best way I know how to do this
      if( string.Compare( e.Url.ToString(), habsWebPage, true ) == 0 )
        Status( "Finished downloading", "Finished downloading webpage" );

        //sneaky way to get rid of the vertical scroll bar in the picture copy of the scrollbar,
        //take a snapshot of the control with the width of the snapshot to be the the width of the control 
        //minus the width of the vertical scrollbar
        //this will work no matter what the user has setup for their windows border sizes,
        //since users can change this value in Windows, you cannot hard code that value otherwise you'll be wrong sometimes
        Rectangle webpageRectangle = habsBrowser.Document.Body.ScrollRectangle;
        webpageRectangle.Width += getScrollbarWidth();
        habsBrowser.Bounds = webpageRectangle;
        Bitmap bitmap = new Bitmap( webpageRectangle.Width - getScrollbarWidth(), webpageRectangle.Height );

        Control control = habsBrowser;
        control.DrawToBitmap( bitmap, webpageRectangle );
        //here's the short form of the above in one line, notice the casting, 
        //HAVE to do that cause DrawToBitmap is private to controls inhereting from Control
        //((Control)habsBrowser).DrawToBitmap( bitmap, webpageRectangle );

        StringFormat stringFormat = new StringFormat( StringFormatFlags.DirectionVertical );
        stringFormat.LineAlignment = StringAlignment.Center;
        stringFormat.Alignment = StringAlignment.Center;
        SizeF cheerSize;
        Font arial = new Font( "Arial", 80, FontStyle.Bold );
        string cheer = "GO HABS GO!";
        //chicken and egg situation, I need a graphics object to get a size measurement to create the Bitmap from a graphics object
        using( Graphics gForMetrics = CreateGraphics() )
          cheerSize = gForMetrics.MeasureString( cheer, arial );
        //have to resize the cheer image due to height of text
        Bitmap habsCheer = new Bitmap( (int)cheerSize.Height, (int)webpageRectangle.Height );
        Graphics g = Graphics.FromImage( habsCheer );
        Brush gradientBrush = new LinearGradientBrush( new Point( 0, 0 ), new Point( habsCheer.Width, habsCheer.Height ), Color.Navy, Color.Red );
        g.Clear( Color.Navy );
        g.FillRectangle( gradientBrush, new Rectangle( 0, 0, habsCheer.Width, habsCheer.Height ) );
        g.DrawString( "GO HABS GO!", arial, Brushes.White, cheerSize.Height / 2, cheerSize.Width, stringFormat );
        habsCheer.RotateFlip( RotateFlipType.Rotate180FlipNone );

        Bitmap combinedPage = new Bitmap( bitmap.Width + habsCheer.Width, bitmap.Height );
        g = Graphics.FromImage( combinedPage );
        g.DrawImage( habsCheer, 0, 0 );
        g.DrawImage( bitmap, habsCheer.Width, 0 );

        if( Directory.Exists( "C:\\temp\\" ) )
          string habsFileName = "C:\\temp\\HabsRule." + ImageFormat.Png.ToString().ToUpper();
          //bitmap.Save( habsFileName, ImageFormat.Png );
          combinedPage.Save( habsFileName, ImageFormat.Png );
          Status( "Saved to " + habsFileName, "Saved to" + Environment.NewLine + habsFileName );

        //Thread.Sleep( 1000 );
        //using the PictureBox image to help redraw the image over and over when the form redraws itself, that's a cheat
        //(why? cause we COULD be doing it by hand instead using the forms Paint method, maybe next time?)
        habsWebPageImage = combinedPage;    //bitmap;
        pageLoaded = true;
        String message = "Getting another piece...";
        Uri docLoaded = e.Url;
        //get the url, replace the path and query slashes with slashes and line feeds to show up ok on the control
        string docUrl = docLoaded.ToString()
          .Replace( docLoaded.PathAndQuery,
            docLoaded.PathAndQuery.Replace( "/", "/" + Environment.NewLine ) );
        Status( message, message + Environment.NewLine + docUrl );
        Thread.Sleep( 2000 );   //artificial hesitation to show something on the control

Now that we have a few more Habs fans, it's time to go grab a coffee and get coding! 


Source Code

Location: Blogs Parent Separator TechTidBits

2 comment(s) so far...

Re: Playing with Graphics/GDI+ to screen scrape the Habs current web page

You just gave me some ideas on how to bypass Fortiguard :)

By Souheil Ibrahim on   Thursday, June 18, 2009 10:44 AM

Re: Playing with Graphics/GDI+ to screen scrape the Habs current web page

Howdy! :> Glad I could help, but I have to ask, what is Fortiguard and why do you want to bypass it? (and uh if the RCMP comes a knock'n at your/my door, we don't know each other, OK?! :> HAHA)

By phenry on   Thursday, June 18, 2009 10:45 AM

Your name:
Gravatar Preview
Your email:
(Optional) Email used only to show Gravatar.
Your website:
Add Comment   Cancel 
Copyright 1999-2012 by   Terms Of Use  Privacy Statement