git branching model

git

git is a tool for the command line to backup and distribute source code. There are multiple beautifully made webpages and GUI tools which help to use it and to make sense out even out of the biggest code bases.

The workflow is simple: Each code base / each project is in its own repository. As soon as you’ve finished a change or a feature you commit (save) your code. You can switch back and forth between commits and can compare the changes you’ve made. If you want to share or backup your repository, you push (upload) it to a git hoster. Other participants in your project can pull (download) your repository, including all commits you’ve made. One repository can have multiple branches (versions of your code), each of those has its own commits. Branches can be merged (united) and created.

As a hoster for git I can recommend github which is perfect for open source projects, and gitlab if you need a lot of private repositories. Both services are free, at the time of writing github has some more useful features (like project management with Kanban) compared to gitlab, but both services do a very good job with hosting and helping you make sense out of large repositories.

As committing, pulling, pushing may be complicated for new users I recommend a GUI application to do those steps for you, until you feel confident to use the command line. One of the best is gitkraken, github even provides its own desktop client.

Git is very handy as a backup tool, but it can also help you to structure different versions of your code, allows you to do code reviews and to allow statistics about the overall progress of the repository.

Branching may be confusing for new users, but if integrated in your workflow in a sensible way it is a very handy tool to allow work on different features and versions of your software in a sensible way. How to name and organize your branches is explained very well in a tutorial on the webpage from atlassian.
Another very well written (but a bit more technical) post to branching concepts can be found on the blog of nvie

continuous integration

In the following article I will use continuous integration for quality control: I want to make sure my projects are building and their unit tests are passing. I’ve configured CI in a project of mine for both the C# and the php part. Here’s how:

C#

I’m using appveyor for this. Its as easy as going to their webpage, and adding your project from github. It will automatically figure out what and how to build. You can customize the actions appveyor executes with an appveyor.yml file in your project root. In the following example I customized appveyor to work with codecov.io to get code coverage of the unit tests:

os: Visual Studio 2017
install:
- set PATH=C:\Program Files (x86)\MSBuild\14.0\Bin;%PATH%
before_build:
  - nuget restore
build:
  project: Famoser.SyncApi.sln
  verbosity: minimal
test_script: 
    - .\packages\OpenCover.4.6.519\tools\OpenCover.Console.exe -register:user -target:"C:\Program Files (x86)\Microsoft Visual Studio 14.0\Common7\IDE\MSTest.exe" -targetargs:"/noresults /noisolation /testcontainer:"".\Famoser.SyncApi.UnitTests\bin\Debug\Famoser.SyncApi.UnitTests.dll" -filter:"+[Famoser.SyncApi]*  -[Famoser.SyncApi]Famoser.SyncApi.Properties.*" -excludebyattribute:*.ExcludeFromCodeCoverage* -hideskipped:All -output:.\SyncApi_coverage.xml -returntargetcode
    - "SET PATH=C:\\Python34;C:\\Python34\\Scripts;%PATH%"
    - pip install codecov
    - codecov -f "SyncApi_coverage.xml"  
# preserve "packages" directory in the root of build folder but will reset it if packages.config is modified
cache:
  - packages -> **\packages.config

php

I’m using travis-ci.org to build & test my application, and codeclimate to check my coding style. As usual with web projects, it needs a little more configuration to get up and running.

codeclimate
For this service, you register your project on the webpage of codeclimate, and create a config file named codeclimate.yml. In it, you must specify the paths of your php files, and the engines you want to look at your code. A ready to use config file may look like this:

engines:
  fixme:
    enabled: true
  phpcodesniffer:
    enabled: true
  phpmd:
    enabled: true  
  duplication:
    enabled: true
    config:
      languages:
      - php
ratings:
  paths: ["Famoser.SyncApi.Webpage/src/**/*"]
exclude_paths: ["Famoser.SyncApi.Webpage/src/public/assets/**/*"]

travis
First, you add a config file called .travis.yml. You need to specify the language and the versions of the language to use for testing. You can also specify scripts which run before / after various events. After the configuration is done, add the project on travis-ci.org.

For my setup with code climate & composer I need some additional config: I need to install composer dependencies and execute a script to push test results to codeclimate. For the connection with codeclimate to work be sure to include

  • an environment variable on travis-ci.org (under travis-ci -> settings) with key CODECLIMATE_REPO_TOKEN and value your_test_reporter_token (you may find this token on codeclimate -> settings -> Test Coverage).
  • a new composer dependency under require-dev: "codeclimate/php-test-reporter": "dev-master"
  • the after_script part in the .travis-yml as specified below.

a ready to use configuration may look like this:

language: php
php:
    - 7.0
    - 7.1
install:
    - cd Famoser.SyncApi.Webpage
    - composer install --dev
after_script:
- ./vendor/bin/test-reporter

For the automatic execution of phpunit tests, configure phpunit. A sample configuration file looks like this:

<?xml version="1.0" encoding="UTF-8"?>
<phpunit bootstrap="vendor/autoload.php" stopOnFailure="false">
    <testsuites>
        <!-- following specifies to executes all files in the tests directory which end in Test.php, for example SimpleTest.php -->
        <testsuite name="Application Test Suite">
            <directory suffix="Test.php">./tests</directory>
        </testsuite>
    </testsuites>
    <php>
        <!-- specify php environment values -->
        <!-- <env name="TESTING_MODE" value="testing"/>-->
    </php>
    <logging>
        <!-- this is important for the code climate connection -->
        <log type="coverage-clover" target="build/logs/clover.xml"/>
    </logging>
    <filter>
        <!-- whitelist all php source files you may use in the process of building & testing -->
        <whitelist>
            <directory suffix=".php">src</directory>
        </whitelist>
    </filter>
</phpunit>

For other services providing CI look up Jenkins and docker

webdesigners

Jason Schuller: http://jason.sc/
A damn good minimalistic webdev. He’s made a small fortune on wordpress themes.
Unfortunately he took down his templates some months ago, it appears that he is working now on a simple pay solutions, which is kind of a pity. If they can still be found somewhere, let me know.

AJ: https://aj.lkn.io/
Makes absolutely awesome templates on https://html5up.net/ (for free) and https://pixelarity.com/ (paid).
He’s even made a simple online webpage creator over at https://carrd.co/ (simple to use)

BlackTie: http://blacktie.co/
Bootstrap webpages.
Not as awesome as from the two designers above, but still pretty well-looking.

Visual Studio Extension: Commands

A command connects the code of your application with the UI of Visual Studio. You can create a command with Add -> New File -> Extensibility -> CustomCommand. In the following context I will refer to it as Command1.
This will create you three files:

Command1Package.cs
The (…)Package.cs file will only be created once for all commands. It constructs them with the Visual Studio Package object. You probably will not have to change this file.

Command1Package.vsct
This file sets the foundation of how your command will appear inside Visual Studio. In short, every item which is displayed has a parent, and sometimes children. There are groups, menus & buttons. Buttons have a group as parent, groups have a menu as parent, menus have group as parent. Each items has a guid, which is kind of the “context” of the element. Below, most of the elements have the same guid (namely Command1PackageCmdSet). Also, each item has a unique id. All used ids & guids are defined in the section at the end of the xml, some will be used later (namely to link the Button to a specific callback).
I’ve cut out all automatically created comments and added some useful ones. Lets take a look at it:

<?xml version="1.0" encoding="utf-8"?>
<CommandTable xmlns="http://schemas.microsoft.com/VisualStudio/2005-10-18/CommandTable" xmlns:xs="http://www.w3.org/2001/XMLSchema">
  <!--includes you need -->
  <Extern href="stdidcmd.h" />
  <Extern href="vsshlids.h" />

  <Commands package="Command1Package">
    <!-- groups together commands. Parent must be menu, children must be commands -->
    <Groups>
      <Group guid="Command1PackageCmdSet" id="MyTopMenuGroup" priority="0x4000">
        <!-- the parent of this group is a visual studio menu, the guid and id can be found in the vast space of the internet -->
        <Parent guid="guidSHLMainMenu" id="IDM_VS_CTXT_ITEMNODE" />
      </Group>

      <Group guid="Command1PackageCmdSet" id="MySubMenuGroup" priority="0x4000">
        <!-- the parent of this group is the menu defined below -->
        <Parent guid="Command1PackageCmdSet" id="MyMenuController" />
      </Group>
    </Groups>
        
    <!-- groups together groups. Parent must be group, children must be group -->
    <Menus>
      <Menu guid="Command1PackageCmdSet" id="MyMenuController" priority="0x0100" type="MenuController">
        <!-- the parent of this menu is the group defined above -->
        <Parent guid="Command1PackageCmdSet" id="MyTopMenuGroup" />
        <Strings>
          <ButtonText>Menu Text</ButtonText>
        </Strings>
      </Menu>
    </Menus>


    <!-- the actual things you can click on. Parent must be group, no children allowed -->
    <Buttons>
      <Button guid="Command1PackageCmdSet" id="CommandId0" priority="0x0100" type="Button">
        <Parent guid="Command1PackageCmdSet" id="MySubMenuGroup" />
        <Icon guid="guidImages" id="bmpPic1" />
        <Strings>
          <ButtonText>Invoke CommandId0</ButtonText>
        </Strings>
      </Button>

      <Button guid="Command1PackageCmdSet" id="CommandId1" priority="0x0100" type="Button">
        <Parent guid="Command1PackageCmdSet" id="MyTopMenuGroup" />
        <!-- the Icon is optional, 
        <Icon guid="guidImages" id="bmpPic1" />
        <Strings>
          <ButtonText>Invoke CommandId1</ButtonText>
        </Strings>
      </Button>
    </Buttons>

    <!--The bitmaps section is used to define the bitmaps that are used for the commands.-->
    <Bitmaps>
      <Bitmap guid="guidImages" href="Resources\TileGeneratorCommand.png" usedList="bmpPic1, bmpPic2, bmpPicSearch, bmpPicX, bmpPicArrows, bmpPicStrikethrough" />
    </Bitmaps>
  </Commands>

  <!-- in this section, all used guids and ids are defined. -->
  <Symbols>
    <!-- the Command1Package context -->
    <GuidSymbol name="Command1Package" value="{aa71ec2a-1f27-46fb-ba00-8da6b0a43c73}" />

    <!-- the Command1PackageCmdSet context -->
    <GuidSymbol name="Command1PackageCmdSet" value="{145c5f09-ce3c-43f7-9948-18f1ba65f860}">
      <IDSymbol name="MyMenuController" value="0x1010" />
      <IDSymbol name="MyTopMenuGroup" value="0x1020" />
      <IDSymbol name="MySubMenuGroup" value="0x1030" />
      <IDSymbol name="CommandId0" value="0x0100" />
      <IDSymbol name="CommandId1" value="0x0200" />
    </GuidSymbol>

    <!-- the guidImages context -->
    <GuidSymbol name="guidImages" value="{99def054-e104-4328-821a-0dbb5d1c20cf}">
      <IDSymbol name="bmpPic1" value="1" />
      <IDSymbol name="bmpPic2" value="2" />
      <IDSymbol name="bmpPicSearch" value="3" />
      <IDSymbol name="bmpPicX" value="4" />
      <IDSymbol name="bmpPicArrows" value="5" />
      <IDSymbol name="bmpPicStrikethrough" value="6" />
    </GuidSymbol>
  </Symbols>
</CommandTable>

Command1.cs
This is the file you launch your logic from.
After you’ve defined the setup of your buttons, commands and groups you need to construct the buttons, and assign callbacks. For each button define a const int in your .cs file:

//use the same value as defined in the .vsct file
public const int CommandId0 = 0x0100;
public const int CommandId1 = 0x0200;

Now, create the Buttons in the constructor, and add them to the Visual Studio environment:

private Command1(Package package)
{
	if (package == null)
	{
		throw new ArgumentNullException("package");
	}

	this.package = package;

	OleMenuCommandService commandService = this.ServiceProvider.GetService(typeof(IMenuCommandService)) as OleMenuCommandService;
	if (commandService != null)
	{
		var command0 = new CommandID(CommandSet, CommandId0);
		var command0Item = new MenuCommand(Command0Callback, command0);
		commandService.AddCommand(command0Item);

		var command1 = new CommandID(CommandSet, CommandId1);
		var command1Item = new MenuCommand(Command1Callback, command1);
		commandService.AddCommand(command1Item);
	}
}

Debug the Application to be sure all worked out correctly.

set up symfony

Prerequisites

  • installed XAMPP or MAMP (apache server & php installation bundles)
  • installed composer (php libraries)
  • installed npm (deploy libraries)

Step 1: install Symfony installer

option 1 (recommended): download the installer, and move it to the directory where your repository should be located

php -r "readfile('https://symfony.com/installer');" &gt; symfony
move symfony D:\Repos
php symfony new my_project

option 2: create the project with composer:

composer create-project symfony/framework-standard-edition my_project

follow the instructions in the console (you will probably need to configure your database & mail settings)

now, update all packages with composer:

composer update

Step 2: set up npm

create a file named package.json, adapt following json to your needs:

{
  "name": "my_project",
  "version": "0.1.0",
  "description": "My Project",
  "repository": {
    "type": "git",
    "url": "git://github.com/famoser/my_project.git"
  },
  "keywords": [
    "office",
	"online"
  ],
  "author": "Florian Moser &lt;git@famoser.ch&gt;",
  "license": "MIT",
  "bugs": {
    "url": "https://github.com/famoser/my_project/issues"
  },
  "homepage": "https://github.com/famoser/my_project",
  "devDependencies": {
    "bower": "^1",
    "browser-sync": "^2",
    "del": "^2",
    "gulp": "^3",
    "gulp-all": "^1",
    "gulp-batch": "^1",
    "gulp-bower": "^0",
    "gulp-clean-css": "^2",
    "gulp-concat": "^2",
    "gulp-less": "^3",
    "gulp-notify": "^2",
    "gulp-rename": "^1",
    "gulp-sass": "^2",
    "gulp-sourcemaps": "^1",
    "gulp-uglify": "^1",
    "gulp-watch": "^4",
    "less-plugin-autoprefix": "^1",
    "rimraf": "^2"
  }
}

run in your console

npm update

Step 3: set up bower

create a file named bower.json, adapt following json to your needs:

{
  "name": "my-project",
  "version": 0.1,
  "homepage": "https://github.com/famoser/my-project",
  "authors": [
    "famoser &lt;git@famoser.ch&gt;"
  ],
  "description": "My Project",
  "keywords": [
    "php"
  ],
  "license": "MIT",
  "ignore": [
    "**/.*",
    "node_modules",
    "bower_components",
    "test",
    "tests"
  ],
  "dependencies": {
    "jquery": "2.2.3",
    "bootstrap": "3.3.6",
    "components-font-awesome": "*"
  }
}

create a file named .bowerrc, with following json content:

{
    "directory": "web/assets/vendor/"
}

to create such a strange filename you can run

copy NUL > .bowerrc

in your console.

Step 4: set up build tools

I’ll use gulp for this.
Create a file named guldfile.js, content must be JavaScript. You’ll find a good starting point here: gist. There is quite a lot going on in this file, the instructions and the configuration is at the top.

Now run in your console

gulp

to build all frontend libraries for the first time.

Step 5: set up source control

  • add a README.MD describing your project
  • add a LICENSE.MD so other users may or may not be allowed to use your work (I’m recommending the MIT license)
  • exclude unnecessary files from git (bower, composer & IDE files)
.idea
node_modules
/web/assets/vendor
symfony-scripts/.git

Step 6: rename your AppBundle

choose a custom name for your application, make it VendorApplicationBundle, so for example GoogleYoutubeBundle.
Files / Folder you need to adapt:

  • The file /src/AppBundle/AppBundle.php
  • The namespaces in all .php files inside /src/AppBundle/**/*.php
  • The folder /src/AppBundle
  • in /app/AppKernel.php you need to register your renamed bundle (just replace the old AppBundle/AppBundle entry)

download youtube video with c#

tl;dr: YoutubeExtractor.Portable

I’ve written a library (or rather made an existing one portable) to download videos & audio from youtube.

Features:

  • Extract all formats & resolutions for a youtube video
  • Download video
  • Convert to audio (mp3 or aac)

currently only audio extraction from an .flv file works. Unfortunately youtube only provides .flv files in low resolutions, which results in low quality audio files. Still, better than nothing.

Code examples are on github.

composer

composer

Composer is a dependency manager for php. It also automatically generates all code necessary for autoloading.

The setup is simple:

place a composer.json in your project folder

{
  "name": "username/repo",
  "type": "framework",
  "license": "MIT",
  "description": "A project which gets things done",
  "keywords": [
    "project keywords", "another keyword"
  ],
  "authors": [
    {
      "name": "Florian Moser",
      "email": "me@myurl.ch",
      "homepage": "http://myurl.ch",
      "role": "Developer"
    }
  ],
  "require": {
    "php": ">=7.0",
    "phpmailer/phpmailer": "5.2.*",
    "famoser/minify": "dev-master"
  },
  "autoload": {
    "psr-4": {
      "username\\myproject\\": "src/"
    }
  },
  "minimum-stability": "dev"
}

Specifiy all packages  you need in the require node.
In the autoload node, define the standard you’re using, your vendor & project prefix, and the folder the source is located.

Let composer do its magic

composer install

This will download / actualize all dependencies specified in the require node (if applicable). Those files will be placed in a vendor folder. Inside this folder, there is a file named autoload.php. This file contains code to autoload all dependecies and additional sources specified in the autoload node. So all you need to do to use the dependencies is to include this autoload.php in your project.

Some other useful commands

composer update

When first running composer install, composer generates a composer.lock file, to ensure everyone executing composer install installs the same versions of the dependencies. composer update deletes this .lock file, and resolves the dependencies again.

You can download the tool from here.

Bower

bower

Bower is a package manager which comes in handy for web frontend frameworks like foundation, bootstrap, jquerysemantic ui, …

Setup is simple:

Place a bower.json in your project root

{
  "name": "project-name",
  "version": 0.1,
  "homepage": "https://url.com/username/repo",
  "authors": [
    "famoser <me@myurl.ch>"
  ],
  "description": "A project which gets things done",
  "keywords": [
    "project keywords", "another keyword"
  ],
  "license": "MIT",
  "ignore": [
    "**/.*",
    "node_modules",
    "bower_components",
    "test",
    "tests"
  ],
  "dependencies": {
    "jquery": "2.2.*",
    "semantic-ui": "*"
  }
}

Let bower do its magic

bower install

This will download / actualize all dependencies specified in the dependencies node.

You may download the tool, after you have set up npm, simply by executing

npm install bower --save-dev

in your console.

npm

npm

The Node Package Manager is a command line javascript package manager. It is powerful if you use build tools like gulp or if you develop server side with javascript.

The setup is simple:

place a package.json in your project folder

{
  "name": "project-name",
  "version": "0.1.0",
  "description": "project-name is a project which gets things done!",
  "repository": {
    "type": "git",
    "url": "git://url.com/username/repo.git"
  },
  "keywords": [
    "project keywords", "another keyword"
  ],
  "author": "Florian Moser <me@myurl.ch>",
  "license": "MIT",
  "bugs": {
    "url": "https://url.com/username/repo/issues"
  },
  "homepage": "https://url.com/username/repo",
  "devDependencies": {
    "gulp": "^3.9.1"
  }
}

Let npm do its magic

npm install

This will download / actualize all dependencies specified in the devDependencies or dependencies node.

Some other useful commands

npm install <package_name>

Install the package into the current directory. Append the flag --save or --save-dev to put the package into your dependencies.

You can download the tool from here.