continuous integration

In the following article I will use continuous integration for quality control: I want to make sure my projects are building and their unit tests are passing. I’ve configured CI in a project of mine for both the C# and the php part. Here’s how:

C#

I’m using appveyor for this. Its as easy as going to their webpage, and adding your project from github. It will automatically figure out what and how to build. You can customize the actions appveyor executes with an appveyor.yml file in your project root. In the following example I customized appveyor to work with codecov.io to get code coverage of the unit tests:

os: Visual Studio 2017
install:
- set PATH=C:\Program Files (x86)\MSBuild\14.0\Bin;%PATH%
before_build:
  - nuget restore
build:
  project: Famoser.SyncApi.sln
  verbosity: minimal
test_script: 
    - .\packages\OpenCover.4.6.519\tools\OpenCover.Console.exe -register:user -target:"C:\Program Files (x86)\Microsoft Visual Studio 14.0\Common7\IDE\MSTest.exe" -targetargs:"/noresults /noisolation /testcontainer:"".\Famoser.SyncApi.UnitTests\bin\Debug\Famoser.SyncApi.UnitTests.dll" -filter:"+[Famoser.SyncApi]*  -[Famoser.SyncApi]Famoser.SyncApi.Properties.*" -excludebyattribute:*.ExcludeFromCodeCoverage* -hideskipped:All -output:.\SyncApi_coverage.xml -returntargetcode
    - "SET PATH=C:\\Python34;C:\\Python34\\Scripts;%PATH%"
    - pip install codecov
    - codecov -f "SyncApi_coverage.xml"  
# preserve "packages" directory in the root of build folder but will reset it if packages.config is modified
cache:
  - packages -> **\packages.config

php

I’m using travis-ci.org to build & test my application, and codeclimate to check my coding style. As usual with web projects, it needs a little more configuration to get up and running.

codeclimate
For this service, you register your project on the webpage of codeclimate, and create a config file named codeclimate.yml. In it, you must specify the paths of your php files, and the engines you want to look at your code. A ready to use config file may look like this:

engines:
  fixme:
    enabled: true
  phpcodesniffer:
    enabled: true
  phpmd:
    enabled: true  
  duplication:
    enabled: true
    config:
      languages:
      - php
ratings:
  paths: ["Famoser.SyncApi.Webpage/src/**/*"]
exclude_paths: ["Famoser.SyncApi.Webpage/src/public/assets/**/*"]

travis
First, you add a config file called .travis.yml. You need to specify the language and the versions of the language to use for testing. You can also specify scripts which run before / after various events. After the configuration is done, add the project on travis-ci.org.

For my setup with code climate & composer I need some additional config: I need to install composer dependencies and execute a script to push test results to codeclimate. For the connection with codeclimate to work be sure to include

  • an environment variable on travis-ci.org (under travis-ci -> settings) with key CODECLIMATE_REPO_TOKEN and value your_test_reporter_token (you may find this token on codeclimate -> settings -> Test Coverage).
  • a new composer dependency under require-dev: "codeclimate/php-test-reporter": "dev-master"
  • the after_script part in the .travis-yml as specified below.

a ready to use configuration may look like this:

language: php
php:
    - 7.0
    - 7.1
install:
    - cd Famoser.SyncApi.Webpage
    - composer install --dev
after_script:
- ./vendor/bin/test-reporter

For the automatic execution of phpunit tests, configure phpunit. A sample configuration file looks like this:

<?xml version="1.0" encoding="UTF-8"?>
<phpunit bootstrap="vendor/autoload.php" stopOnFailure="false">
    <testsuites>
        <!-- following specifies to executes all files in the tests directory which end in Test.php, for example SimpleTest.php -->
        <testsuite name="Application Test Suite">
            <directory suffix="Test.php">./tests</directory>
        </testsuite>
    </testsuites>
    <php>
        <!-- specify php environment values -->
        <!-- <env name="TESTING_MODE" value="testing"/>-->
    </php>
    <logging>
        <!-- this is important for the code climate connection -->
        <log type="coverage-clover" target="build/logs/clover.xml"/>
    </logging>
    <filter>
        <!-- whitelist all php source files you may use in the process of building & testing -->
        <whitelist>
            <directory suffix=".php">src</directory>
        </whitelist>
    </filter>
</phpunit>

For other services providing CI look up Jenkins and docker

Visual Studio Extension: Commands

A command connects the code of your application with the UI of Visual Studio. You can create a command with Add -> New File -> Extensibility -> CustomCommand. In the following context I will refer to it as Command1.
This will create you three files:

Command1Package.cs
The (…)Package.cs file will only be created once for all commands. It constructs them with the Visual Studio Package object. You probably will not have to change this file.

Command1Package.vsct
This file sets the foundation of how your command will appear inside Visual Studio. In short, every item which is displayed has a parent, and sometimes children. There are groups, menus & buttons. Buttons have a group as parent, groups have a menu as parent, menus have group as parent. Each items has a guid, which is kind of the “context” of the element. Below, most of the elements have the same guid (namely Command1PackageCmdSet). Also, each item has a unique id. All used ids & guids are defined in the section at the end of the xml, some will be used later (namely to link the Button to a specific callback).
I’ve cut out all automatically created comments and added some useful ones. Lets take a look at it:

<?xml version="1.0" encoding="utf-8"?>
<CommandTable xmlns="http://schemas.microsoft.com/VisualStudio/2005-10-18/CommandTable" xmlns:xs="http://www.w3.org/2001/XMLSchema">
  <!--includes you need -->
  <Extern href="stdidcmd.h" />
  <Extern href="vsshlids.h" />

  <Commands package="Command1Package">
    <!-- groups together commands. Parent must be menu, children must be commands -->
    <Groups>
      <Group guid="Command1PackageCmdSet" id="MyTopMenuGroup" priority="0x4000">
        <!-- the parent of this group is a visual studio menu, the guid and id can be found in the vast space of the internet -->
        <Parent guid="guidSHLMainMenu" id="IDM_VS_CTXT_ITEMNODE" />
      </Group>

      <Group guid="Command1PackageCmdSet" id="MySubMenuGroup" priority="0x4000">
        <!-- the parent of this group is the menu defined below -->
        <Parent guid="Command1PackageCmdSet" id="MyMenuController" />
      </Group>
    </Groups>
        
    <!-- groups together groups. Parent must be group, children must be group -->
    <Menus>
      <Menu guid="Command1PackageCmdSet" id="MyMenuController" priority="0x0100" type="MenuController">
        <!-- the parent of this menu is the group defined above -->
        <Parent guid="Command1PackageCmdSet" id="MyTopMenuGroup" />
        <Strings>
          <ButtonText>Menu Text</ButtonText>
        </Strings>
      </Menu>
    </Menus>


    <!-- the actual things you can click on. Parent must be group, no children allowed -->
    <Buttons>
      <Button guid="Command1PackageCmdSet" id="CommandId0" priority="0x0100" type="Button">
        <Parent guid="Command1PackageCmdSet" id="MySubMenuGroup" />
        <Icon guid="guidImages" id="bmpPic1" />
        <Strings>
          <ButtonText>Invoke CommandId0</ButtonText>
        </Strings>
      </Button>

      <Button guid="Command1PackageCmdSet" id="CommandId1" priority="0x0100" type="Button">
        <Parent guid="Command1PackageCmdSet" id="MyTopMenuGroup" />
        <!-- the Icon is optional, 
        <Icon guid="guidImages" id="bmpPic1" />
        <Strings>
          <ButtonText>Invoke CommandId1</ButtonText>
        </Strings>
      </Button>
    </Buttons>

    <!--The bitmaps section is used to define the bitmaps that are used for the commands.-->
    <Bitmaps>
      <Bitmap guid="guidImages" href="Resources\TileGeneratorCommand.png" usedList="bmpPic1, bmpPic2, bmpPicSearch, bmpPicX, bmpPicArrows, bmpPicStrikethrough" />
    </Bitmaps>
  </Commands>

  <!-- in this section, all used guids and ids are defined. -->
  <Symbols>
    <!-- the Command1Package context -->
    <GuidSymbol name="Command1Package" value="{aa71ec2a-1f27-46fb-ba00-8da6b0a43c73}" />

    <!-- the Command1PackageCmdSet context -->
    <GuidSymbol name="Command1PackageCmdSet" value="{145c5f09-ce3c-43f7-9948-18f1ba65f860}">
      <IDSymbol name="MyMenuController" value="0x1010" />
      <IDSymbol name="MyTopMenuGroup" value="0x1020" />
      <IDSymbol name="MySubMenuGroup" value="0x1030" />
      <IDSymbol name="CommandId0" value="0x0100" />
      <IDSymbol name="CommandId1" value="0x0200" />
    </GuidSymbol>

    <!-- the guidImages context -->
    <GuidSymbol name="guidImages" value="{99def054-e104-4328-821a-0dbb5d1c20cf}">
      <IDSymbol name="bmpPic1" value="1" />
      <IDSymbol name="bmpPic2" value="2" />
      <IDSymbol name="bmpPicSearch" value="3" />
      <IDSymbol name="bmpPicX" value="4" />
      <IDSymbol name="bmpPicArrows" value="5" />
      <IDSymbol name="bmpPicStrikethrough" value="6" />
    </GuidSymbol>
  </Symbols>
</CommandTable>

Command1.cs
This is the file you launch your logic from.
After you’ve defined the setup of your buttons, commands and groups you need to construct the buttons, and assign callbacks. For each button define a const int in your .cs file:

//use the same value as defined in the .vsct file
public const int CommandId0 = 0x0100;
public const int CommandId1 = 0x0200;

Now, create the Buttons in the constructor, and add them to the Visual Studio environment:

private Command1(Package package)
{
	if (package == null)
	{
		throw new ArgumentNullException("package");
	}

	this.package = package;

	OleMenuCommandService commandService = this.ServiceProvider.GetService(typeof(IMenuCommandService)) as OleMenuCommandService;
	if (commandService != null)
	{
		var command0 = new CommandID(CommandSet, CommandId0);
		var command0Item = new MenuCommand(Command0Callback, command0);
		commandService.AddCommand(command0Item);

		var command1 = new CommandID(CommandSet, CommandId1);
		var command1Item = new MenuCommand(Command1Callback, command1);
		commandService.AddCommand(command1Item);
	}
}

Debug the Application to be sure all worked out correctly.

download youtube video with c#

tl;dr: YoutubeExtractor.Portable

I’ve written a library (or rather made an existing one portable) to download videos & audio from youtube.

Features:

  • Extract all formats & resolutions for a youtube video
  • Download video
  • Convert to audio (mp3 or aac)

currently only audio extraction from an .flv file works. Unfortunately youtube only provides .flv files in low resolutions, which results in low quality audio files. Still, better than nothing.

Code examples are on github.

mvvm pattern in c#

This is an implementation in C# of the MVVM pattern as described before.

In this implementation I use the MVVM light toolkit, and some patterns I’ve already described: inversion of control and dependecy injection.

First, lets start with the model:

    public class NoteModel : ObservableObject
    {
        private string _note;
        public string Note
        {
            get { return _note; }
            set { Set(ref _note, value); }
        }

        private bool _isFinished;
        public bool IsFinished
        {
            get { return _isFinished; }
            set { Set(ref _isFinished, value); }
        }
    }

As you can see, when assigning to a property, I use a method provided my the MVVM light library: Set(). This method assigns the new value to the property, and fires the IPropertyChanged event, if the new value differs from the old. The IPropertyChanged event allows the view to refresh automatically.

Now, lets look at the view:

<Page DataContext="{Binding MainViewModel, Source={StaticResource Locator}}">
    <Grid>
        <Grid.RowDefinitions>
            <RowDefinition Height="Auto" />
            <RowDefinition Height="10" />
            <RowDefinition Height="*" />
        </Grid.RowDefinitions>
        <Grid Grid.Row="0">
            <Grid.ColumnDefinitions>
                <ColumnDefinition Width="*" />
                <ColumnDefinition Width="Auto" />
            </Grid.ColumnDefinitions>
            <TextBox Text="{Binding NewNote, Mode=TwoWay, UpdateSourceTrigger=PropertyChanged}" />
            <Button Command="{Binding AddNoteCommand}" >
                <Button.Content>
                    <SymbolIcon Symbol="Add" />
                </Button.Content>
            </Button>
        </Grid>
        <ListView ItemsSource="{Binding Notes}" />
    </Grid>
</Page>

I’ve removed all styling informations so everything is easier to understand. Take a look at DataContext: MainViewModel is a property of Locator, and this property is bound to the view. Take a look at the TextBox to see what this means: the Text property is filled with another {Binding NewNote...}, and NewNote is a property of the MainViewModel from above.

There are three types of binding in this example:

  • read binding: The ListView binds itself to the Notes property.
  • read & write in the Textbox: The Mode=TwoWay ensures, user input is written into the property automatically by XAML magic.
  • command binding: the Button binds itself to a property called AddNoteCommand, this property implements the ICommand interface. We’ll see about that.

I recommend you to take a look at this in Visual Studio, this helps you to understand all the other magic XAML stuff.

One last thing to check out: the {StaticResource Locator} part in the DataContext, you’ll find the definition the App.xaml:

<viewModels:ViewModelLocator x:Key="Locator" />

which leads to (in the Presentation project, ViewModels folder):

    public class ViewModelLocator : ViewModelLocatorBase
    {
        public ViewModelLocator()
        {
            ServiceLocator.SetLocatorProvider(() => SimpleIoc.Default);
            SimpleIoc.Default.Register<IPlatformInformations, PlatformInformations>();
        }
    }

and this leads to (in the View project, ViewModels folder)

    public class ViewModelLocatorBase
    {
        public ViewModelLocatorBase()
        {
            ServiceLocator.SetLocatorProvider(() => SimpleIoc.Default);
            SimpleIoc.Default.Register<IDataService, DataService>();
            SimpleIoc.Default.Register<MainViewModel>();
        }

        public MainViewModel MainViewModel => SimpleIoc.Default.GetInstance<MainViewModel>();
    }

Now there is the MainViewModel property!
The SimpleIoc part is an implementation of the Inversion of Control pattern, and provided by the Mvvm light library.

Now the ViewModel.
The constructor part:

    public class MainViewModel : ViewModelBase
    {
        private readonly IDataService _dataService;

        //inversion of control: SimpleIoc will contruct this object with the registered IDataService implementation
        public MainViewModel(IDataService dataService)
        {
            _dataService = dataService;

            _addNoteCommand = new RelayCommand(AddNote, () => CanAddNote);
            _deleteNoteCommand = new RelayCommand<NoteModel>(DeleteNote);

            if (IsInDesignMode)
            {
                Notes = new ObservableCollection<NoteModel>()
                {
                    new NoteModel()
                    {
                        Note = "Hello World",
                        IsFinished = false
                    }
                };
            }
            else
            {
                Initialize();
            }
        }

        private async void Initialize()
        {
            Notes = await _dataService.GetTasks();
        }

        private ObservableCollection<NoteModel> _notes;
        public ObservableCollection<NoteModel> Notes
        {
            get { return _notes; }
            set { Set(ref _notes, value); }
        }
    }

MainViewModel inherits from the ViewModelBase, a class provided by the MVVM light library. It has some nice tools integrated, for example the IsInDesignMode propery.

The ObservableCollection takes care of the necessary events firing, so the view can refresh itself if necessary.

The NewNote property is the one filled by the Textbox. Depending on the input, one can create a new note. So every time the value changes (Set returns true if the value has changed), the should reevaluate if a new note could be created.


        private string _newNote;
        public string NewNote
        {
            get { return _newNote; }
            set
            {
                if (Set(ref _newNote, value))
                    _addNoteCommand.RaiseCanExecuteChanged();
            }
        }

The commands:

        private readonly RelayCommand _addNoteCommand;
        public ICommand AddNoteCommand => _addNoteCommand;

        private bool CanAddNote => !string.IsNullOrEmpty(NewNote);

        private void AddNote()
        {
            Notes.Add(new NoteModel() { Note = NewNote });
            NewNote = "";
        }

        private readonly RelayCommand<NoteModel> _deleteNoteCommand;
        public ICommand DeleteNoteCommand => _deleteNoteCommand;

        private void DeleteNote(NoteModel model)
        {
            Notes.Remove(model);
        }

You can pass an object directly to the command, as done with the DeleteNoteCommand, with the CommandParameter property in the view.

Now start the application and play with it a little bit.
You’ll notice two things:

  • Without any code in view, the displayed items are always up to date. Even some basic animations already work
  • The logic is completely divided from the display, you may use the exact same ViewModel in an Android or iOS application

repository pattern

This pattern is an application of the separation of concerns principle. It basically separates the consumer from the supplier of data.

For the consumer of an RSS feed it is irrelevant if the feed is read out from the storage, or downloaded from the internet. This is where the Repository Pattern jumps into action; it provides a simple API for the consumer, and takes care of resolving the data from the best source available.

A small example (c#):

internal class RssRepository
{
    public List<RssItem> GetRssItems()
    {
        if (DeviceHelper.HasInternet)
        {
            return RssHelper.DownloadFeedItems();
        }
        else
        {
            return StorageHelper.RetrieveCachedFeedItems();
        }
    }
}

async await

async-await is a very important concept from C#. It allows you to execute code in another thread than from the calling one; this results in a responsive UI even while  long background tasks are running.

Some examples:

/* ## Example 1 ## */
public async Task<bool> Execute()
{
    //long running CPU bound operation
    await Task.Run(() => {
    for (int i = 0; i != 10000000; ++i)
        ;
    });

    //long running I/O Operation
    using (var client = new HttpClient())
    {
        await client.GetStringAsync(url);
    }
    return true;
}

//call this method
var response = await Execute();

/* ## Example 2 ## */
//remove the async-await if not needed, and return directly the Task.
public Task<string> Execute2()
{
    //long running I/O Operation
    using (var client = new HttpClient())
    {
        return client.GetStringAsync(url);
    }
}

//call this method
var response = await Execute2();

/* ## Example 3 ## */
//add the async keyword to event handlers to use them asynchronusly
public async void OnTapped()
{
    //long running I/O Operation
    using (var client = new HttpClient())
    {
        await client.GetStringAsync(url);
    }
}

From my own experiences, the simpler the expressions, the better. If it looks wrong or overcomplicated to you, it probably is. I recommend to take a look at Stephen Cleary’s Blog (specifically at async-await) for extensive information and tutorials about async-await.

inversion of control

A very handy pattern for large projects. Think of it as an enhancement from the dependency injection pattern.

When using dependency injection, the caller of a class provides the implementations of the requirements.
This gets cumbersome with classes which have a lot of requirements, and large code bases where you may not even know where the implementation is, or how it is called.

Inversion Of Control to the rescue: With this pattern, the framework provides the implementations.

As the implementation is a bit more complex, I’ll only explain the concept:

  • Create a helper, which can construct classes with requirements (non-empty contructor with interfaces)
  • Register the implementations you want to use against the helper at a central place in your code
  • Retrieve all classes which have requirements over this helper

An Example in pseudo-php code:

class IoCContainer extends Singleton
{
    private $registeredInterfaces = array();
    public function register($interface, $implementation)
    {
        $this->registeredInterfaces[$interface] = $implementation;
    }

    public function construct($object)
    {
        $arguments = get_needed_arguments($object);
        //match each argument with an implementation, and construct the object
        return $instance;
    }
}

now at the start of your application you register the implementations:

IoCContainer::getInstance()->register("IProgress",new ProgressService());
class Workflow
{
    public function __construct(IProgress $progressService) { }

    public function doStuff() { }
}
IoCContainer::getInstance()->construct("Workflow")->doStuff();

Now, you do not want to write an IoCContainer yourself. Not because it is hard, but because it has been made before, and tested before. For C# I can recommend the SimpleIoC Container from the MVVMLight Package: MVVM Light (you can get it over NuGet). There are similar implementations for most other languages.

dependecy injection

This is one of my favorite patterns out there! Use this if you have a large codebase or multiple consumers of a library.

Use case: You have some code, and multiple Consumers (Webpage, Phone Application & Desktop Application) which want to use the contained Logic (so you do not copy paste the same code three times). Now you want to show progress for a long running operation in the DownloadLargeFileWorkflow. What to do:

  • You create a IProgress interface
  • Each of your Consumers implements this interface
  • You pass the instance which implements this interface to the shared code.

Ready! Now you can use the same code for all three very different consumers.

Now how to implement this?

public interface IProgress
{
    void ShowProgress();
    void HideProgress();
}

public class DownloadLargeFileWorkflow
{
    private IProgress _progress;
    pulic DownloadLargeFileWorkflow(IProgress progress)
    {
        _progress = progress;
    }

    public void Execute()
    {
        _progress.ShowProgress();
        //do long running operation
        _progress.HideProgress();
    }
}

In your consumer specific code you can now construct the DownloadLargeFileWorkflow with your specific implementation of the IProgress interface.

singleton

If you use this, only do so for logging. This pattern is widely described as an anti-pattern, don’t tell me I did not warn you: Patterns I Hate #1: SingletonSingleton (…) you’re bringing me down. Still, I do like this pattern because if its simplicity, and Dependency Injection or the Factory pattern is for small projects sometimes just overkill.

This pattern ensures there is only one instance of a class active.
This is useful for logging, where you only need one logging instance.

The implementation itself is quite simple:

php (for C# see here: github):

    
class Singleton
{
    //overwrite those two methods to "disable" them
    private function __clone() {}
    private function __construct() {}

    //our saved instances
    protected static $instances = array();

    /**
     * @return static
     */
    final public static function getInstance()
    {
        $classname = get_called_class();
        if (!isset(static::$instances[$classname])) {
            static::$instances[$classname] = new static();
        }
        return static::$instances[$classname];
    }
}

To use this pattern you simply inherit from the singleton class, and then call MyOneAndOnly::getInstance() (php) or MyOneAndOnly.Instance.HelloWorld (C#)

class MyOneAndOnly extends Singleton 
{
    public function helloWorld() { return "hello"; }
}

echo MyOneAndOnly::getInstance()->helloWorld(); //prints "hello"

Only use this pattern if your application does not depend on the data of the Singleton; you only pass data to the object, but never retrieve any. This scenario is true for a Logger, or for a Stopwatch to test the performance of your application. For any other scenario fo not use this, as it makes your code harder to understand and to test