AForge.NET Framework and a custom Screenshot Video Source


I’ve been playing with the AForge framework which is an open source project currently licensed under the LGPL and written by Andrew Kirillov. It’s located at http://code.google.com/p/aforge/. This framework does a lot of stuff, but the two main pieces that interest me are the web cam functionality and the motion detection classes that he’s put together (that are very easy to use). I’ve been using it to develop a program I’ve dubbed Timershot.Net that replaces (and adds) the functionality of the Timershot power toy for Windows XP that no longer works under Windows Vista or Windows 7 (http://www.microsoft.com/windowsxp/downloads/powertoys/xppowertoys.mspx). I’ve built my program using the AForge VideoCaptureDevice which inherits from IVideoSource. So I thought, why not implement the same interface and create a custom IVideoDevice for a screen shot that I could just plugin. Here is my first pass (this isn’t done, doesn’t have all of the features or error handling implemented but it does work). Aside from the ScreenCaptureDevice class, I’m using one other class I have that handles actually getting the screen/window pictures from the OS. I will include both here. When I refine this class, I will update the code snippet here (or I’ll provide a link to the update post). I should note, the class is written for the .Net Framework 4.0, but the AForge libraries currently target the .Net Framework 2.0 (you’ll notice I used the auto implemented properties in the new version of VB). Apologies also, some of the spacing in the code below is a little off.

Imports System
Imports System.Drawing
Imports System.Threading
Imports AForge.Video

''' <summary>
''' A custom IVideoSource device to capture screenshots from Windows.
''' </summary>
Public Class ScreenCaptureDevice
    Implements AForge.Video.IVideoSource
    '*********************************************************************************************************************
    '
    '             Class:  ScreenCaptureDevice
    '      Initial Date:  02/04/2011
    '      Last Updated:  02/07/2011
    '     Programmer(s):  Blake Pell
    '
    '*********************************************************************************************************************
    Private _threadCapture As New System.Threading.Thread(AddressOf ScreenCapture)

    Private Sub ScreenCapture()
        Do
            SyncLock Me
                _framesReceived += 1
                Dim bm As Bitmap
                Select Case Me.ScreenshotType
                    Case ScreenshotTypes.ActiveWindow
                        bm = Screenshot.GetScreenshotCurrentWindow
                    Case ScreenshotTypes.PrimaryDesktop
                        bm = Screenshot.GetScreenshotPrimaryScreen
                    Case Else
                        ' Get's rid of the compiler warning
                        bm = New Bitmap(0, 0)
                End Select
                If bm Is Nothing Then
                    ' Don't know why, the but the Bitmap here ends up null in some cases.  If it is null, just get out for now.
                    Exit Sub
                Else
                    ' The Bitmap shouldn't be null, raise the NewFrame event then free the resources.
                    RaiseEvent NewFrame(Nothing, New AForge.Video.NewFrameEventArgs(bm))
                    bm.Dispose()
                    bm = Nothing
                End If
                ' This is to keep the memory usage down.  The CLR should handle and free memory when the OS needs it but
                ' a lot of people will complain when this quickly uses upwards of 500-700MB of RAM depending on how fast
                ' screenshots are pulled in.  This will keep the memory usage down but will create some additional overhead.
                If Me.AutoCallGarbageCollect = True Then
                    GC.Collect()
                End If
            End SyncLock
            Threading.Thread.Sleep(Me.Interval)
        Loop
    End Sub

    ''' <summary>
    ''' Whether or not to automatically call the garbage collection.  This will keep the memory usage down but may impact
    ''' performance as the 
    ''' </summary>
    Public Property AutoCallGarbageCollect As Boolean = True
    
    ''' <summary>
    ''' The interval in milleseconds that a screenshot will be taken.  The default is 1000 (1 second)
    ''' </summary>
    Property Interval As Integer = 1000

    ''' <summary>
    ''' The different types of screenshots that can be taken.
    ''' </summary>
    Public Enum ScreenshotTypes
        ''' <summary>
        ''' The window that currently has focus.
        ''' </summary>
        ActiveWindow
        ''' <summary>
        ''' The primary desktop window.  If multiple monitors are used, this will be the main monitor.
        ''' </summary>
        PrimaryDesktop
    End Enum

    ''' <summary>
    ''' The type of screenshot to take.
    ''' </summary>
    ''' <remarks>
    ''' Choosing just the active window will have a smaller memory footprint and resources usage, both on the local CPU and
    ''' in sending the specified image to the FTP server.
    ''' </remarks>
    Public Property ScreenshotType As ScreenshotTypes = ScreenshotTypes.ActiveWindow

    Private _bytesReceived As Integer = 0

    ''' <summary>
    ''' Not implemented, will return 0.
    ''' </summary>
    Public ReadOnly Property BytesReceived As Integer Implements AForge.Video.IVideoSource.BytesReceived
        Get
            Return _bytesReceived
        End Get
    End Property

    Private _framesReceived As Integer = 0

    ''' <summary>
    ''' Not implemented, will return 0.
    ''' </summary>
    Public ReadOnly Property FramesReceived As Integer Implements AForge.Video.IVideoSource.FramesReceived
        Get
            Return _framesReceived
        End Get
    End Property

    ''' <summary>
    ''' If the class is currently capturing the screen images and the threads are still active.
    ''' </summary>
    Public ReadOnly Property IsRunning As Boolean Implements AForge.Video.IVideoSource.IsRunning
        Get
            Return _threadCapture.IsAlive
        End Get
    End Property

    ''' <summary>
    ''' Event notifying that a new Bitmap is ready. 
    ''' </summary>
    ''' <param name="sender"></param>
    ''' <param name="eventArgs"></param>
    Public Event NewFrame(ByVal sender As Object, ByVal eventArgs As AForge.Video.NewFrameEventArgs) Implements AForge.Video.IVideoSource.NewFrame

    ''' <summary>
    ''' Not implemented by this class.
    ''' </summary>
    ''' <param name="sender"></param>
    ''' <param name="reason"></param>
    ''' <remarks></remarks>
    Public Event PlayingFinished(ByVal sender As Object, ByVal reason As AForge.Video.ReasonToFinishPlaying) Implements AForge.Video.IVideoSource.PlayingFinished

    Public Sub SignalToStop() Implements AForge.Video.IVideoSource.SignalToStop
        Me.Stop()
    End Sub

    ''' <summary>
    ''' Not implemented by this class.
    ''' </summary>
    Public ReadOnly Property Source As String Implements AForge.Video.IVideoSource.Source
        Get
            Return ""
        End Get
    End Property

    ''' <summary>
    ''' Starts the screen capture process.
    ''' </summary>
    Public Sub Start() Implements AForge.Video.IVideoSource.Start
        _threadCapture = New System.Threading.Thread(AddressOf ScreenCapture)
        _threadCapture.Start()
    End Sub

    ''' <summary>
    ''' Stops the screen capture process.
    ''' </summary>
    Public Sub [Stop]() Implements AForge.Video.IVideoSource.Stop
        _threadCapture.Abort()
    End Sub

    Public Event VideoSourceError(ByVal sender As Object, ByVal eventArgs As AForge.Video.VideoSourceErrorEventArgs) Implements AForge.Video.IVideoSource.VideoSourceError

    Public Sub WaitForStop() Implements AForge.Video.IVideoSource.WaitForStop

    End Sub

End Class
Imports System.Windows.Forms
Imports System.Drawing
Imports System.Drawing.Imaging
Imports System.Runtime.InteropServices
Imports System.ComponentModel

Namespace Argus.Windows
    ''' <summary>
    ''' This class can be used to take a screenshot of the current window, the desktop, multiple desktops or specified
    ''' sections of the desktoip.  It returns all screenshots as a System.Drawing.Bitmap
    ''' </summary>
    ''' <remarks>
    ''' 
    ''' TODO:  - Add support for Graphics.CopyFromScreen and BitBlt
    '''        - Return as byte array
    ''' 
    ''' <code>
    ''' ' Example of Graphics.CopyFromScreen with a MemoryStream
    ''' Dim ms As New System.IO.MemoryStream
    ''' Dim bm As New System.Drawing.Bitmap(Screen.PrimaryScreen.Bounds.Width, Screen.PrimaryScreen.Bounds.Height, PixelFormat.Format32bppArgb)
    ''' Dim screenShot As Graphics = Graphics.FromImage(bm)
    ''' screenShot.CopyFromScreen(Screen.PrimaryScreen.Bounds.X, Screen.PrimaryScreen.Bounds.Y, 0, 0, Screen.PrimaryScreen.Bounds.Size, CopyPixelOperation.SourceCopy)
    ''' bm.Save(ms, ImageFormat.Jpeg)
    ''' Dim imageBytes As Byte() = ms.ToArray
    ''' Return imageBytes
    ''' </code>
    ''' 
    ''' Dependencies:
    ''' <list>
    ''' System.Drawing
    ''' </list>
    ''' </remarks>
    Public Class Screenshot

        '*********************************************************************************************************************
        '
        '             Class:  Screenshot
        '      Initial Date:  09/16/2007
        '      Last Updated:  01/24/2012
        '     Programmer(s):  Blake Pell, bpell@indiana.edu
        '
        '*********************************************************************************************************************
        ''' <summary>
        ''' The method that is used to take the screenshot.
        ''' </summary>
        ''' <remarks></remarks>
        Enum ScreenshotMethod
            ''' <summary>
            ''' The BitBlt method using the BitBlt Windows API in the gdi32 library file.  Using the Windows API may provide
            ''' benefits but also may break with future OS releases.
            ''' </summary>
            ''' <remarks></remarks>
            BitBlt
            ''' <summary>
            ''' The CopyFromScreen method uses .Net's Graphics class to take a screenshot.  This method uses all managed code
            ''' from the .Net Framework and should be sheletered from changes in the OS.
            ''' </summary>
            ''' <remarks>
            ''' At the writing of this code, the Graphics.CopyFromScreen method had some issues with copying pixels in regards
            ''' to Aero's transparency which is why both the BitBlt and this method are provided.
            ''' </remarks>
            CopyFromScreen
        End Enum

        <DllImport("gdi32")> _
        Public Shared Function BitBlt(ByVal hDestDC As IntPtr, ByVal X As Integer, ByVal Y As Integer, ByVal nWidth As Integer, ByVal nHeight As Integer, ByVal hSrcDC As IntPtr, ByVal SrcX As Integer, ByVal SrcY As Integer, ByVal Rop As Integer) As Boolean
        End Function

        <DllImport("user32.dll")> _
        Private Shared Function GetWindowDC(ByVal hwnd As IntPtr) As Integer
        End Function

        <DllImport("user32.dll")> _
        Public Shared Function GetDesktopWindow() As IntPtr
        End Function

        <DllImport("user32.dll")> _
        Public Shared Function GetForegroundWindow() As IntPtr
        End Function

        <DllImport("user32.dll")> _
        Private Shared Function ReleaseDC(ByVal hWnd As IntPtr, ByVal hDc As IntPtr) As IntPtr
        End Function

        <DllImport("user32")> _
        Private Shared Function GetWindowRect(ByVal hWnd As IntPtr, ByRef lpRect As RECT) As Integer
        End Function

        Private Const SRCCOPY As Integer = &HCC0020

        ''' <summary>
        ''' Rectable structure to pass to the Windows API's
        ''' </summary>
        <StructLayout(LayoutKind.Sequential)> _
        Private Structure RECT
            Dim Left As Integer
            Dim Top As Integer
            Dim Right As Integer
            Dim Bottom As Integer
        End Structure

        ''' <summary>
        ''' Takes a screenshot of the primary screen and returns it as a Sysem.Drawing.Bitmap.
        ''' 
        ''' This function uses the BitBlt Windows API to take the screenshot.
        ''' 
        ''' </summary>
        Public Shared Function GetScreenshotPrimaryScreen() As Bitmap
            Dim g As Graphics
            Dim hdcDest As IntPtr = IntPtr.Zero
            Dim desktopHandleDC As IntPtr = IntPtr.Zero
            Dim desktopHandle As IntPtr = Screenshot.GetDesktopWindow()
            Dim bmp As Bitmap = New Bitmap(Screen.PrimaryScreen.Bounds.Width, Screen.PrimaryScreen.Bounds.Height)
            bmp = New Bitmap(Screen.PrimaryScreen.Bounds.Width, Screen.PrimaryScreen.Bounds.Height)
            g = Graphics.FromImage(bmp)
            desktopHandleDC = Screenshot.GetWindowDC(desktopHandle)
            hdcDest = g.GetHdc
            BitBlt(hdcDest, 0, 0, Screen.PrimaryScreen.Bounds.Width, Screen.PrimaryScreen.Bounds.Height, desktopHandleDC, 0, 0, SRCCOPY)
            g.ReleaseHdc(hdcDest)
            ReleaseDC(desktopHandle, desktopHandleDC)
            g.Dispose() : g = Nothing
            Return bmp
        End Function

        ''' <summary>
        ''' Returns a list of bitmaps that contain a bitmap for every display screen.  This method uses the Graphics.CopyFromScreen
        ''' method.  
        ''' </summary>
        Public Shared Function GetScreenshotAllScreens() As List(Of Bitmap)
            Dim bmpList As New List(Of Bitmap)

            For Each sc As Screen In Screen.AllScreens
                Dim g As Graphics
                Dim bmp As Bitmap = New Bitmap(sc.Bounds.Width, sc.Bounds.Height)
                g = Graphics.FromImage(bmp)
                g.CopyFromScreen(sc.Bounds.Left, sc.Bounds.Top, 0, 0, New Size(sc.Bounds.Width, sc.Bounds.Height))
                bmpList.Add(bmp)
                g.Dispose() : g = Nothing
            Next
            Return bmpList
        End Function

        ''' <summary>
        ''' Takes a screenshot of the current window and return it as a System.Drawing.Bitmap.
        ''' 
        ''' This function uses the BitBlt Windows API to take the screenshot.
        ''' </summary>
        Public Shared Function GetScreenshotCurrentWindow() As Bitmap
            Dim g As Graphics
            Dim hdcDest As IntPtr = IntPtr.Zero
            Dim windowHandleDC As IntPtr = IntPtr.Zero
            Dim windowHandle As IntPtr = Screenshot.GetForegroundWindow
            Dim windowRect As RECT
            Dim bmp As Bitmap
            If GetWindowRect(windowHandle, windowRect) Then
                bmp = New Bitmap(windowRect.Right - windowRect.Left, windowRect.Bottom - windowRect.Top)
            Else
                Return Nothing
            End If
            g = Graphics.FromImage(bmp)
            windowHandleDC = Screenshot.GetWindowDC(windowHandle)
            hdcDest = g.GetHdc
            BitBlt(hdcDest, 0, 0, bmp.Width, bmp.Height, windowHandleDC, 0, 0, SRCCOPY)
            g.ReleaseHdc(hdcDest)
            ReleaseDC(windowHandle, windowHandleDC)
            g.Dispose() : g = Nothing
            Return bmp
        End Function

        ''' <summary>
        ''' Takes a screenshot associated with the given handle (be it a window or control) and return it
        ''' as a System.Drawing.Bitmap.
        ''' 
        ''' This function uses the BitBlt Windows API to take the screenshot.
        ''' </summary>
        ''' <param name="handle"></param>
        Public Shared Function GetScreenshotByHandle(ByVal handle As IntPtr) As Bitmap
            Dim g As Graphics
            Dim hdcDest As IntPtr = IntPtr.Zero
            Dim windowHandleDC As IntPtr = IntPtr.Zero
            Dim windowRect As RECT
            Dim bmp As Bitmap
            If GetWindowRect(handle, windowRect) Then
                bmp = New Bitmap(windowRect.Right - windowRect.Left, windowRect.Bottom - windowRect.Top)
            Else
                Return Nothing
            End If
            g = Graphics.FromImage(bmp)
            windowHandleDC = Screenshot.GetWindowDC(handle)
            hdcDest = g.GetHdc
            BitBlt(hdcDest, 0, 0, bmp.Width, bmp.Height, windowHandleDC, 0, 0, SRCCOPY)
            g.ReleaseHdc(hdcDest)
            ReleaseDC(handle, windowHandleDC)
            g.Dispose() : g = Nothing
            Return bmp
        End Function

        ''' <summary>
        ''' Takes a screenshot of the specified location and returns it as a System.Drawing.Bitmap.
        ''' 
        ''' This function uses the BitBlt Windows API to take the screenshot.
        ''' 
        ''' </summary>
        ''' <param name="rectLoc"></param>
        Public Shared Function GetScreenshotByLocation(ByVal rectLoc As Rectangle) As Bitmap
            Dim g As Graphics
            Dim hdcDest As IntPtr = IntPtr.Zero
            Dim windowHandleDC As IntPtr = IntPtr.Zero
            Dim windowRect As RECT
            windowRect.Left = rectLoc.Left
            windowRect.Right = rectLoc.Right
            windowRect.Top = rectLoc.Top
            windowRect.Bottom = rectLoc.Bottom
            Dim bmp As New Bitmap(windowRect.Right - windowRect.Left, windowRect.Bottom - windowRect.Top)
            g = Graphics.FromImage(bmp)
            windowHandleDC = Screenshot.GetDesktopWindow
            hdcDest = g.GetHdc
            BitBlt(hdcDest, 0, 0, bmp.Width, bmp.Height, windowHandleDC, windowRect.Left, windowRect.Top, SRCCOPY)
            g.ReleaseHdc(hdcDest)
            ReleaseDC(Screenshot.GetDesktopWindow, windowHandleDC)
            g.Dispose() : g = Nothing
            Return bmp
        End Function
    End Class
End Namespace