continuous integration

In the following article I will use continuous integration for quality control: I want to make sure my projects are building and their unit tests are passing. I’ve configured CI in a project of mine for both the C# and the php part. Here’s how:

C#

I’m using appveyor for this. Its as easy as going to their webpage, and adding your project from github. It will automatically figure out what and how to build. You can customize the actions appveyor executes with an appveyor.yml file in your project root. In the following example I customized appveyor to work with codecov.io to get code coverage of the unit tests:

os: Visual Studio 2017
install:
- set PATH=C:\Program Files (x86)\MSBuild\14.0\Bin;%PATH%
before_build:
  - nuget restore
build:
  project: Famoser.SyncApi.sln
  verbosity: minimal
test_script: 
    - .\packages\OpenCover.4.6.519\tools\OpenCover.Console.exe -register:user -target:"C:\Program Files (x86)\Microsoft Visual Studio 14.0\Common7\IDE\MSTest.exe" -targetargs:"/noresults /noisolation /testcontainer:"".\Famoser.SyncApi.UnitTests\bin\Debug\Famoser.SyncApi.UnitTests.dll" -filter:"+[Famoser.SyncApi]*  -[Famoser.SyncApi]Famoser.SyncApi.Properties.*" -excludebyattribute:*.ExcludeFromCodeCoverage* -hideskipped:All -output:.\SyncApi_coverage.xml -returntargetcode
    - "SET PATH=C:\\Python34;C:\\Python34\\Scripts;%PATH%"
    - pip install codecov
    - codecov -f "SyncApi_coverage.xml"  
# preserve "packages" directory in the root of build folder but will reset it if packages.config is modified
cache:
  - packages -> **\packages.config

php

I’m using travis-ci.org to build & test my application, and codeclimate to check my coding style. As usual with web projects, it needs a little more configuration to get up and running.

codeclimate
For this service, you register your project on the webpage of codeclimate, and create a config file named codeclimate.yml. In it, you must specify the paths of your php files, and the engines you want to look at your code. A ready to use config file may look like this:

engines:
  fixme:
    enabled: true
  phpcodesniffer:
    enabled: true
  phpmd:
    enabled: true  
  duplication:
    enabled: true
    config:
      languages:
      - php
ratings:
  paths: ["Famoser.SyncApi.Webpage/src/**/*"]
exclude_paths: ["Famoser.SyncApi.Webpage/src/public/assets/**/*"]

travis
First, you add a config file called .travis.yml. You need to specify the language and the versions of the language to use for testing. You can also specify scripts which run before / after various events. After the configuration is done, add the project on travis-ci.org.

For my setup with code climate & composer I need some additional config: I need to install composer dependencies and execute a script to push test results to codeclimate. For the connection with codeclimate to work be sure to include

  • an environment variable on travis-ci.org (under travis-ci -> settings) with key CODECLIMATE_REPO_TOKEN and value your_test_reporter_token (you may find this token on codeclimate -> settings -> Test Coverage).
  • a new composer dependency under require-dev: "codeclimate/php-test-reporter": "dev-master"
  • the after_script part in the .travis-yml as specified below.

a ready to use configuration may look like this:

language: php
php:
    - 7.0
    - 7.1
install:
    - cd Famoser.SyncApi.Webpage
    - composer install --dev
after_script:
- ./vendor/bin/test-reporter

For the automatic execution of phpunit tests, configure phpunit. A sample configuration file looks like this:

<?xml version="1.0" encoding="UTF-8"?>
<phpunit bootstrap="vendor/autoload.php" stopOnFailure="false">
    <testsuites>
        <!-- following specifies to executes all files in the tests directory which end in Test.php, for example SimpleTest.php -->
        <testsuite name="Application Test Suite">
            <directory suffix="Test.php">./tests</directory>
        </testsuite>
    </testsuites>
    <php>
        <!-- specify php environment values -->
        <!-- <env name="TESTING_MODE" value="testing"/>-->
    </php>
    <logging>
        <!-- this is important for the code climate connection -->
        <log type="coverage-clover" target="build/logs/clover.xml"/>
    </logging>
    <filter>
        <!-- whitelist all php source files you may use in the process of building & testing -->
        <whitelist>
            <directory suffix=".php">src</directory>
        </whitelist>
    </filter>
</phpunit>

For other services providing CI look up Jenkins and docker

set up symfony

Prerequisites

  • installed XAMPP or MAMP (apache server & php installation bundles)
  • installed composer (php libraries)
  • installed npm (deploy libraries)

Step 1: install Symfony installer

option 1 (recommended): download the installer, and move it to the directory where your repository should be located

php -r "readfile('https://symfony.com/installer');" &gt; symfony
move symfony D:\Repos
php symfony new my_project

option 2: create the project with composer:

composer create-project symfony/framework-standard-edition my_project

follow the instructions in the console (you will probably need to configure your database & mail settings)

now, update all packages with composer:

composer update

Step 2: set up npm

create a file named package.json, adapt following json to your needs:

{
  "name": "my_project",
  "version": "0.1.0",
  "description": "My Project",
  "repository": {
    "type": "git",
    "url": "git://github.com/famoser/my_project.git"
  },
  "keywords": [
    "office",
	"online"
  ],
  "author": "Florian Moser &lt;git@famoser.ch&gt;",
  "license": "MIT",
  "bugs": {
    "url": "https://github.com/famoser/my_project/issues"
  },
  "homepage": "https://github.com/famoser/my_project",
  "devDependencies": {
    "bower": "^1",
    "browser-sync": "^2",
    "del": "^2",
    "gulp": "^3",
    "gulp-all": "^1",
    "gulp-batch": "^1",
    "gulp-bower": "^0",
    "gulp-clean-css": "^2",
    "gulp-concat": "^2",
    "gulp-less": "^3",
    "gulp-notify": "^2",
    "gulp-rename": "^1",
    "gulp-sass": "^2",
    "gulp-sourcemaps": "^1",
    "gulp-uglify": "^1",
    "gulp-watch": "^4",
    "less-plugin-autoprefix": "^1",
    "rimraf": "^2"
  }
}

run in your console

npm update

Step 3: set up bower

create a file named bower.json, adapt following json to your needs:

{
  "name": "my-project",
  "version": 0.1,
  "homepage": "https://github.com/famoser/my-project",
  "authors": [
    "famoser &lt;git@famoser.ch&gt;"
  ],
  "description": "My Project",
  "keywords": [
    "php"
  ],
  "license": "MIT",
  "ignore": [
    "**/.*",
    "node_modules",
    "bower_components",
    "test",
    "tests"
  ],
  "dependencies": {
    "jquery": "2.2.3",
    "bootstrap": "3.3.6",
    "components-font-awesome": "*"
  }
}

create a file named .bowerrc, with following json content:

{
    "directory": "web/assets/vendor/"
}

to create such a strange filename you can run

copy NUL > .bowerrc

in your console.

Step 4: set up build tools

I’ll use gulp for this.
Create a file named guldfile.js, content must be JavaScript. You’ll find a good starting point here: gist. There is quite a lot going on in this file, the instructions and the configuration is at the top.

Now run in your console

gulp

to build all frontend libraries for the first time.

Step 5: set up source control

  • add a README.MD describing your project
  • add a LICENSE.MD so other users may or may not be allowed to use your work (I’m recommending the MIT license)
  • exclude unnecessary files from git (bower, composer & IDE files)
.idea
node_modules
/web/assets/vendor
symfony-scripts/.git

Step 6: rename your AppBundle

choose a custom name for your application, make it VendorApplicationBundle, so for example GoogleYoutubeBundle.
Files / Folder you need to adapt:

  • The file /src/AppBundle/AppBundle.php
  • The namespaces in all .php files inside /src/AppBundle/**/*.php
  • The folder /src/AppBundle
  • in /app/AppKernel.php you need to register your renamed bundle (just replace the old AppBundle/AppBundle entry)
composer

composer

Composer is a dependency manager for php. It also automatically generates all code necessary for autoloading.

The setup is simple:

place a composer.json in your project folder

{
  "name": "username/repo",
  "type": "framework",
  "license": "MIT",
  "description": "A project which gets things done",
  "keywords": [
    "project keywords", "another keyword"
  ],
  "authors": [
    {
      "name": "Florian Moser",
      "email": "me@myurl.ch",
      "homepage": "http://myurl.ch",
      "role": "Developer"
    }
  ],
  "require": {
    "php": ">=7.0",
    "phpmailer/phpmailer": "5.2.*",
    "famoser/minify": "dev-master"
  },
  "autoload": {
    "psr-4": {
      "username\\myproject\\": "src/"
    }
  },
  "minimum-stability": "dev"
}

Specifiy all packages  you need in the require node.
In the autoload node, define the standard you’re using, your vendor & project prefix, and the folder the source is located.

Let composer do its magic

composer install

This will download / actualize all dependencies specified in the require node (if applicable). Those files will be placed in a vendor folder. Inside this folder, there is a file named autoload.php. This file contains code to autoload all dependecies and additional sources specified in the autoload node. So all you need to do to use the dependencies is to include this autoload.php in your project.

Some other useful commands

composer update

When first running composer install, composer generates a composer.lock file, to ensure everyone executing composer install installs the same versions of the dependencies. composer update deletes this .lock file, and resolves the dependencies again.

You can download the tool from here.

php autoloading

In php, before you can use a class or method inside a file, you have to include the file first.

As it is cumbersome and error-prone to do this manually, you should use autoloading. You have to meet two requirements for this to work best:

  • Use a valid Objective Oriented approach: for example don’t globally define methods, but group similar methods in classes (example: define a public static function convertToInches in the class UnitConverter instead of just defining it globally)
  • Use namespaces: Use namespaces corresponding to the folder structure of your project. Learn about this at PSR-4

The autoloading itself is quite simple and saves you a lot of time. Place the code below at the entry point of your application.

/*** class loader ***/
function load_my_classes($class) {
    // project-specific namespace prefix
    $prefix = 'Vendor\\Project\\';

    // does the class use the namespace prefix?
    $len = strlen($prefix);
    if (strncmp($prefix, $class, $len) === 0) {
		// get the relative class name
                $relative_class = substr($class, $len);
    
		// replace namespace separators with directory separators in the relative class name
		// append .php
		$file = __DIR__ . "/" . str_replace('\\', '/', $relative_class) . '.php';

		// if the file exists, require it
		if (file_exists($file)) {
			require $file;
		} else {    
			throw new Exception("Unable to load class: " . $class . " @ path: " . $file);
		}
	} else {
		//as the class uses an unknown namespace prefix, we ignore this one. An other autoloader can take care of this.
	}
});

/*** register the class loader ***/
spl_autoload_register('spl_autoload_register');

Now, whenever php has to use a class it does not know yet, it calls all methods registered by spl_autoload_register (in our case this is load_my_classes), until the class it included.
If it has called all registered methods, but still the class is not found, your application terminates (obviously).

If you’ve followed the naming conventions as specified in PSR-4, you can even take this a step further with composer. Besides other very useful things, composer does automatically generate the autoloading code. It’s worth giving it a try.

mvc pattern in php

This is an implementation in php of the MVC pattern as described before.

First, we need to ensure all requests go to one file only, the index.php file. There we will construct our controller, and run the application. We do this by placing a correctly configured .htaccess file in the root directory of our webpage. I’ve commented out the commands, feel free to take a look.

The actual code in the index.php file is quite simple:

// (skipped code for including files)

//construct the controller
$controller = new HomeController($_SERVER["REQUEST_URI"]);
//execute the application
echo $controller->execute();

That was easy. Let’s take a look at our controller:

class HomeController {
	private $params;
	
	/**
	 * HomeController constructor.
	 *
	 * @param $params : the request URL
	 */
	public function __construct($params) {
		$this->params = $params;
	}
	
	/**
	 * execute the application
	 *
	 * @return string
	 */
	public function execute() {
		if ($this->params == "/") {
			$view = new HomeView("hallo welt!");
			return $view->renderHome();
		}
		else {
			if ($this->params == "/famoser") {
				$view = new HomeView();
				$person = new PersonModel("famoser", "01.01.1990");
				return $view->renderPerson($person);
			}
			else {
				if ($this->params == "/famoser/multiple") {
					$view = new HomeView();
					$persons = array();
					for ($i = 0; $i &amp;amp;amp;lt; 5; $i++) $persons[] = new PersonModel("famoser (" . $i . ")", "01.01.1990"); return $view->renderMultiplePersons($persons);
				}
				else {
					$view = new HomeView("not found");
					$view->addKeyValue("url", $this->params);
					return $view->renderError();
				}
			}
		}
	}
}

As you can see, the controller checks if the $params variable is a valid url. If it is, it contructs an object called the HomeView, and passes models and text to it. Then it lets the HomeView render the templates, and returns the received data.

Let take a look at a simplified HomeView (full code at github):

/**
* a simplyfied HomeView, I've removed the constructor and other for this example irrelevant code
*/
class HomeView {
	private $message;
	private $keyValues;

	public function getMessage() {
		return $this->message;
	}
	
	/**
	 * use this function to pass simple data to the view
	 */
	public function addKeyValue($key, $value) {
		$this->keyValues[$key] = $value;
	}
	
	/**
	 * retrieves values added by addKeyValue from the controller
	 */
	public function getValue($key) {
		return $this->keyValues[$key];
	}
	
	/**
	 * render the home template
	 */
	public function renderHome() {
		return $this->includeFile("index");
	}
	
	/**
	 * render the person template
	 */
	public function renderPerson(PersonModel $person) {
		$this->person = $person;
		return $this->includeFile("person");
	}

	/**
	 * render multiple persons
	 */
	public function renderMultiplePersons(array $persons) {
		$result = "";
		foreach ($persons as $person) {
			$this->person = $person;
			$result .= "

" . $this->includeFile("person") . "

";
		}
		return $result;
	}
	
	/**
	 * renders a file, writes the output it produces into a variable and returns that.
	 */
	private function includeFile($fileName) {
                //shorted
                include $filename;

	}
}

As you can see, the HomeView basically serves as a container for all data it needs to display. As the controller calls a function as renderHome, it loads a specific template and returns the rendered content to the controller.
Lastly, lets take a look at a template file:

if ($this instanceof HomeView) {
	echo $this->getMessage();
}

As I’m including the file from the HomeView, the context in this file ($this) is still the view. Now I can call the methods from the view to retrieve the data I have to render to human readable information.

What advantages has this approach?

  • reuse of templates is made easy: from one template, I can include others by calling functions provided by the view object (for example renderMultiplePersons)
  • a clear separation of view & logic: the controller does not care how the data he provides is displayed, he simply constructs the view with the appropriate data, and the view does not care where the data comes from, and just loads the appropriate templates.

inversion of control

A very handy pattern for large projects. Think of it as an enhancement from the dependency injection pattern.

When using dependency injection, the caller of a class provides the implementations of the requirements.
This gets cumbersome with classes which have a lot of requirements, and large code bases where you may not even know where the implementation is, or how it is called.

Inversion Of Control to the rescue: With this pattern, the framework provides the implementations.

As the implementation is a bit more complex, I’ll only explain the concept:

  • Create a helper, which can construct classes with requirements (non-empty contructor with interfaces)
  • Register the implementations you want to use against the helper at a central place in your code
  • Retrieve all classes which have requirements over this helper

An Example in pseudo-php code:

class IoCContainer extends Singleton
{
    private $registeredInterfaces = array();
    public function register($interface, $implementation)
    {
        $this->registeredInterfaces[$interface] = $implementation;
    }

    public function construct($object)
    {
        $arguments = get_needed_arguments($object);
        //match each argument with an implementation, and construct the object
        return $instance;
    }
}

now at the start of your application you register the implementations:

IoCContainer::getInstance()->register("IProgress",new ProgressService());
class Workflow
{
    public function __construct(IProgress $progressService) { }

    public function doStuff() { }
}
IoCContainer::getInstance()->construct("Workflow")->doStuff();

Now, you do not want to write an IoCContainer yourself. Not because it is hard, but because it has been made before, and tested before. For C# I can recommend the SimpleIoC Container from the MVVMLight Package: MVVM Light (you can get it over NuGet). There are similar implementations for most other languages.

singleton

If you use this, only do so for logging. This pattern is widely described as an anti-pattern, don’t tell me I did not warn you: Patterns I Hate #1: SingletonSingleton (…) you’re bringing me down. Still, I do like this pattern because if its simplicity, and Dependency Injection or the Factory pattern is for small projects sometimes just overkill.

This pattern ensures there is only one instance of a class active.
This is useful for logging, where you only need one logging instance.

The implementation itself is quite simple:

php (for C# see here: github):

    
class Singleton
{
    //overwrite those two methods to "disable" them
    private function __clone() {}
    private function __construct() {}

    //our saved instances
    protected static $instances = array();

    /**
     * @return static
     */
    final public static function getInstance()
    {
        $classname = get_called_class();
        if (!isset(static::$instances[$classname])) {
            static::$instances[$classname] = new static();
        }
        return static::$instances[$classname];
    }
}

To use this pattern you simply inherit from the singleton class, and then call MyOneAndOnly::getInstance() (php) or MyOneAndOnly.Instance.HelloWorld (C#)

class MyOneAndOnly extends Singleton 
{
    public function helloWorld() { return "hello"; }
}

echo MyOneAndOnly::getInstance()->helloWorld(); //prints "hello"

Only use this pattern if your application does not depend on the data of the Singleton; you only pass data to the object, but never retrieve any. This scenario is true for a Logger, or for a Stopwatch to test the performance of your application. For any other scenario fo not use this, as it makes your code harder to understand and to test