Chris Webb's BI Blog

Analysis Services, MDX, PowerPivot, DAX and anything BI-related

Archive for the ‘Power BI’ Category

New Power BI Q&A Functionality Released: Optimisation In The Browser

with one comment

Seems like another new bit of Power BI functionality got released today: the ability to optimize your data model for Q&A in the browser. Here’s the link to the docs:

http://office.microsoft.com/en-us/office-365-business/power-bi-q-a-optimize-a-power-bi-workbook-cloud-modeling-HA104226408.aspx?redir=0

Previously, the ability to add synonyms to your model to improve the results you got from Q&A was only available in Excel on the desktop, inside the Power Pivot window. Now you can do this, as well as new stuff like add phrasings (described here) and view usage reports, in your Power BI site.

I won’t repeat what the docs say about the actual functionality, but this seems to be yet more evidence that Excel on the desktop is no longer the central hub for Power BI. If this is the case, this is a massive strategic change, and I can understand why it has happened: the need for the ‘right’ version of Excel on the desktop is a massive roadblock for Power BI adoption, especially in enterprise accounts (see also Jen Underwood’s comments on this from yesterday). Maybe now it’s BI in the browser instead?

Written by Chris Webb

July 15, 2014 at 8:44 pm

Posted in Power BI, Q&A

New Power BI Features Shown At WPC

with 9 comments

OK, so I’m not at WPC this year but I have just watched this video of Scott Guthrie’s session “The Cloud for Modern Business”. If you’re interested in seeing some new Power BI features take a look at the demo by James Phillips, general manager for Power BI, starting at 21:20:

http://www.digitalwpc.com/Videos/Pages/Videos.aspx?g=4d5ef40c-dc5b-426d-9a7a-8dd6274bb42b#fbid=gaOpLt1jjcA

Some of the new things I noticed:

  • 21:40 – a nice shot of one of the new Power BI dashboards first announced at the PASS BA Conference earlier this year. You can see several new types of visualisation such as treemaps, radar charts and gauges (gauges? GAUGES? Shhh, don’t tell Stephen Few).
  • 22:33 – a list of out-of-the box data sources is shown from which new models can be created. They include: Salesforce, MS Dynamics, Facebook, Google Analytics, Twitter, and Upload Excel.
  • 22:50 – data is imported from Salesforce in the browser. This isn’t happening in Excel on the desktop, folks, it’s in the browser. This is significant!
  • 23:10 – another new visualisation shown, a doughnut chart (if that’s the right term). I see names of people from the Power Query team in the data.
  • 24:50 – a Q&A analysis is pinned to the dashboard
  • 25:50 – much is made of the fact that the dashboard is touch-enabled
  • 25:55 – “Partner Solution Packs” are announced. This sounds important! It seems to be referring to the Salesforce demo earlier, and these solution packs are said to include: data, connectivity to the data sources, visualisation and interactive reports. So it sounds like Microsoft are going to encourage data vendors (or other sources of data) to build these solution packs on top of Power BI as pre-packaged analytical apps. Probably a good idea.
  • 26:15 – editing a dashboard in the browser and swapping one visualisation for another. Again, the HTML 5 browser based editing experience – we haven’t seen Excel once in this demo.
  • 27:55 – “If there was ever a partner opportunity, this is it”. Again much emphasis here. Seems like these new Power BI features, especially the solution packs, are aimed at giving partners incentives to sell and customise Power BI (something which they have not had up to now, to be honest).

Oh, and you probably already heard that Azure Machine Learning is now in public preview. Check out the docs and samples here. I wouldn’t be surprised if there was some integration between this and Power BI to come too.

Written by Chris Webb

July 14, 2014 at 11:53 pm

Posted in Power BI

Power Query Book Published!

with 8 comments

Looking for some summer holiday (or winter holiday, depending on which hemisphere you live in) reading? If so, may I suggest my new Power Query book? “Power Query for Power BI and Excel” is available now from the Apress site, Amazon.com, Amazon.co.uk and all good bookstores.

Power Query for Power BI and Excel Cover Image

It’s an introductory level book. It covers all of the stuff you can do in the UI, it has a chapter on M, and it goes into a reasonable amount of detail on more advanced topics; it is not a 500-page exhaustive guide to the product. I’ve focused on readability and teaching the fundamentals of Power Query rather than every looking at every obscure M function, but at the same time if you’ve already used Power Query I think there’ll be plenty of material in there you’ll find interesting.

Now for the bad news: the book is out-of-date already, although not by much. One of the best things about Power Query is the monthly release cycle; unfortunately that makes writing a book on it a bit of a nightmare. I started off writing in January and had to deal with lots of added functionality and changes to the UI over the next few months; I had to retake pretty much all of the screenshots as a result. The published version of the book is based on the version of Power Query that was released in early June rather than the current version. Hopefully you can forgive this – the differences are minor – but it’s a good reason to buy the book as soon as you can! I want to do a second edition in a year’s time once (if?) the release cycle slows down.

I’ve been teased a bit for blogging and teaching so much about Power Query recently, so the final thing I want to say here is why an old corporate BI/SSAS guy like me is getting so excited about a self-service ETL tool. Well, the main reason is that Power Query is a great piece of software. It does what it does very well; it does useful things rather than what the marketing guys/analysts/journalists think is hot in BI; it is easy to use but at the same time is flexible enough for the advanced user to do really complex stuff; it is updated regularly based on feedback from its users. I only wish all Microsoft software was this good… Honestly, I wouldn’t be able to motivate myself to blog and write about Power Query if I didn’t think it was cool, and even though it hasn’t been hyped in the same way as other parts of the Power BI stack it is nonetheless the part that people get excited about when I show them Power BI. It’s not just me either – every day I see positive comments like Greg Low’s here. I think it is as important, if not more important, than Power Pivot and I think it will be a massive success.

Oh, and did I mention that I’m also teaching a Power Query course in London later this year….?

Written by Chris Webb

July 12, 2014 at 3:09 pm

Posted in Books, Power BI, Power Query

Implementing A Basic LIKE/Wildcard Search Function In Power Query

with 3 comments

Last week someone asked me whether it was possible to do the equivalent of a SQL LIKE filter in Power Query. Unfortunately there isn’t a function to do this in the standard library but, as always, it is possible to write some M code to do this. Here’s what I came up while I was waiting around at the stables during my daughter’s horse-riding lesson. At the moment it only supports the % wildcard character; also I can’t guarantee that it’s the most efficient implementation or indeed 100% bug-free, but it seems to work fine as far as I can see…

let

    Like = (Phrase as text, Pattern as text) => 

let

    //Split pattern up into a list using % as a delimiter

    PatternList = Text.Split(Pattern, "%"),

    //if the first character in the pattern is %

    //then the first item in the list is an empty string

    StartsWithWc = (List.First(PatternList)=""),

    //if the last character in the pattern is %

    //then the last item in the list is an empty string

    EndsWithWc = (List.Last(PatternList)=""),

    //if the first character is not %

    //then we have to match the first string in the pattern

    //with the opening characters of the phrase

    StartsTest = if (StartsWithWc=false) 

       then Text.StartsWith(Phrase, List.First(PatternList)) 

       else true,

    //if the last item is not %

    //then we have to match the final string in the pattern

    //with the final characters of the phrase

    EndsText = if (EndsWithWc=false) 

       then Text.EndsWith(Phrase, List.Last(PatternList)) 

       else true,

    //now we also need to check that each string in the pattern appears 

    //in the correct order in the phrase

    //and to do this we need to declare a function PhraseFind

    PhraseFind = (Phrase as text, SearchString as list) =>

    let

     //does the first string in the pattern appear in the phrase?

     StringPos = Text.PositionOf(Phrase, SearchString{0}, Occurrence.First),

     PhraseFindOutput = 

                 if

                 //if string not find then return false 

                 (StringPos=-1) 

                 then false 

                 else if

                 //we have found the string in the pattern, and

                 //if this is the last string in the pattern, return true

                 List.Count(SearchString)=1

                 then true

                 else

                 //if it isn't the last string in the pattern

                 //test the next string in the pattern by removing

                 //the first string from the pattern list

                 //and all text up to and including the string we have found in the phrase

                 (true and

                 @PhraseFind(

                 Text.RemoveRange(Phrase, 0, StringPos + Text.Length(SearchString{0})),

                 List.RemoveRange(SearchString, 0, 1)))

     in

      PhraseFindOutput,

    //return true if we have passed all tests    

    Output = StartsTest and EndsText and PhraseFind(Phrase, PatternList) 

in

    Output

in

    Like

 

Using the following test data:

image

I can run the following query:

let

    Source = Excel.CurrentWorkbook(){[Name="Phrases"]}[Content],

    ChangedType = Table.TransformColumnTypes(Source,{{"Phrases", type text}}),

    InsertedCustom = Table.AddColumn(ChangedType, "Test", each Like([Phrases],"%cat%sat%mat%"))

in

    InsertedCustom

 

And get this output:

image

You can download the sample workbook here.

I know the Power Query team have been asked for this several times already, but it would be really useful if we could package up functions like this and make it easy to share them publicly with other Power Query users…

Written by Chris Webb

May 27, 2014 at 2:19 pm

Posted in Power BI, Power Query

More Thoughts On The New Power BI Features

with 18 comments

My last post on the new Power BI features announced at the PASS BA Conference was not much more than a list of bullet points written during the keynote. Now that the conference is over and I’ve had a bit more time to reflect, I thought I would try to come to some conclusions about what they all mean. And, of course, indulge in some wild speculation too.

Cross-platform

Microsoft’s new-found enthusiasm for working on multiple platforms is clear from two things: first, the announcement that the iOS Power BI app will be coming soon (though we should not forget that this was promised a long, long time ago and is very late), followed by native apps for other mobile platforms; and secondly the work that has gone into the HTML 5 version of Power View. Indeed, the latter was demonstrated using Google Chrome to underline the point. Of course this is the only commercially sane direction to take but it’s very welcome nonetheless.

Power View new features

The cool new Data Exploration features in Power View, which allow you to edit existing Power View reports – creating new graphs and tables and merging existing ones – are I think only going to be available in HTML 5 Power View running inside a Power BI site. The same goes for the new time series forecasting functionality. The demos also seemed to make a point of the touch-friendly interface. Now I can’t imagine that Excel Power View will move from Silverlight to HTML 5 any time soon (the Office team are notoriously conservative when it comes to big changes like this) so maybe we should assume that, going forward, the main focus will be the use of Power View inside a Power BI site to build reports and dashboards rather than Power View inside Excel on the desktop? Maybe the HTML 5 version of Power View will be what is used in the touch-optimised version of Office that is slated to appear this summer? Who knows. I liked what I saw though and these additions will go a long way towards boosting Power View’s credibility as a client tool.

Time series forecasting

It seemed like Microsoft had abandoned ‘data mining for the masses’ after SSAS data mining failed to take off, but clearly not. The time series forecasting functionality seems very easy to use (you can read more about it here and here) and I got the impression that other algorithms might be added soon – maybe Project Sage is relevant here? Another question to ask is whether ease-of-use was the real reason why ‘data mining for the masses’ failed to take off the first time around? It might have been part of the reason but another factor must surely be that business users don’t trust predictions when they don’t understand how they are made, while the data scientists and statisticians are already using other tools that give them a lot more control for forecasting.

RIP PerformancePoint?

The first reaction of several people at the BA Conference (me included) to the new dashboard and KPI creation features in Power BI sites was that this was the replacement for PerformancePoint. Personally I never liked PerformancePoint much and rarely used it, and it doesn’t seem that MS has had much enthusiasm for it in recent years. I don’t think we’ll ever see it in the cloud either. Killing it off would have the added benefit of removing a client tool from a stack that has a confusing number of client tool options right now. However I got the impression that the dashboard and KPI creation features in Power BI sites, as demoed, were fairly basic and they may not be much more than widgets that can be placed on the Power BI home screen with nothing to tie them together, but I don’t know enough to judge properly. I think it might be better to think of Power View as the place to build dashboards.

Integration with old-school SQL Server BI

The ability of Power BI sites to host SSRS reports, and for Power View to connect back to SSAS on-prem, is an important bridge between what most of Microsoft’s BI customers are actually using and the new world of cloud-based Power BI that Microsoft is promoting. For me this was the biggest announcement of all. Will these customers be interested in buying Power BI licences for their users if SSRS and on-prem SSAS is all they want to use though? I don’t know, but I assume that this will also enable mobile access to SSRS and SSAS via the Power BI mobile apps, so that will be a plus for some customers, and all the new Power View functionality makes it quite an attractive web-based reporting tool for SSAS users. The per user cost of a Power BI licence might make it too expensive to buy if all you want are web-based dashboards and mobile support but customers who are already checking out Power BI for self-service will be more likely to buy because of this.

Q&A

New features in Q&A were not mentioned in the keynote on Thursday, but on Friday afternoon I saw an interesting session that detailed some of the new functionality coming in Q&A later this year including the support for phrasing that is mentioned in this blog post. It’s clear that Q&A is getting a lot of love at MS and technically it is very impressive. I’m still not totally convinced that this is something people actually want or will use but I’m less cynical than I was. I also smell a lot of consultancy money in building and tuning models for Q&A if it does get popular (the session showed that there are a lot more features coming that will help with tuning). If I understand correctly, the output from Q&A is basically a Power View report which can then be edited manually if you wish; this means that Q&A should not be thought of as a standalone tool but as one of the ways that you can start to build a Power View report, and Q&A will benefit from all of the new features that are going to be added to Power View.

Timelines

Apart from the time series forecasting, which is available now, ‘this summer’ was the most common response to all questions about when the new stuff would be available. Maybe all of these features will be released when the introductory pricing period for Power BI ends? Or in time for the Worldwide Partner Conference in mid July, as Jen Underwood suggests? Hopefully it won’t be in the middle of my summer holidays.

Written by Chris Webb

May 11, 2014 at 10:11 pm

Posted in Power BI

Power BI Announcements At The PASS BA Conference

with 8 comments

This morning I was wondering whether we’d see any cool new stuff announced at the PASS BA Conference this year, or whether we’d see the same old Power BI demos yet again. It turns out there were a whole load of announcements, some of them very cool indeed – and here’s a brief summary:

  • A native Power BI mobile for iOS will be available by the end of the summer. Other platforms (which means Android I guess) will come soon after that.
  • SSRS will be available in Power BI sites and will be able to connect back to on-prem data sources. This is big, in my opinion – it will be very attractive for a lot of existing MS BI customers. Also the way to get SSRS reports on mobile?
  • Power View in Power BI will be able to connect back to SSAS on premises (just Tabular though, or Multidimensional too?)
  • A new KPI editor in the Power BI site will allow you not only create KPIs but arrange them to create dashboards. This looks like the replacement for PerformancePoint.
  • Time series forecasting is available in Power View online now. It’s available in charts and you just forecast by dragging the chart forward; outliers can be corrected easily. Will need to check this out later.
  • Power View is getting a treemap visualisation.
  • Power View is getting a new Data Exploration mode that allows you to edit reports in the browser. This has a lot of cool new stuff, such as the ability to drag data points out of existing charts to create new charts.

Lots to follow up on there… more blog posts on this coming soon, I promise!

Written by Chris Webb

May 8, 2014 at 5:24 pm

Posted in Events, Power BI

Web Services And POST Requests In Power Query

with 9 comments

In my recent post on web services in Power Query I mentioned that while the Power Query Web.Contents() function generates a GET request by default, you can make it generate a POST request by specifying the Content option. Since this is a useful thing to be able to do I thought I’d put together a detailed example of how this works.

For my example I’m going to use the Bing Maps Elevations API, which allows you to get the elevation in metres for a set of geographic locations. You can read the documentation here:

http://msdn.microsoft.com/en-us/library/jj158961.aspx

As it says at the bottom of the page, if you have a large number of locations to pass to the web service you can do so by passing them as a comma delimited list of latitudes and longitudes using a POST request. Here’s the code for a Power Query query that generates a list of latitudes and longitudes that stretches across the UK from North Wales in the west to the Wash in the east and finds the elevation for each point:

let

    //Insert your Bing Maps API key here

    BingMapsKey = "xxxx",

    //Latitude

    Latitude = 53,

    //Generate a list of longitudes

    Longitudes = List.Numbers(-4.5,66,0.1),

    //Generate a list of latitudes and longitudes

    PointList = List.Transform(Longitudes, 

        each  Number.ToText(Latitude) & "," & Number.ToText(_)),

    //Turn this list to comma delimited text

    PointListText = Text.Combine(PointList,","),

    //Add the text "points=" to the beginning

    PostContents = "points=" & PointListText,

    //Call the Elevations web service

    GetElevations = Web.Contents(

                    "http://dev.virtualearth.net/REST/v1/Elevation/List?key=" & BingMapsKey,

                    [Content=Text.ToBinary(PostContents)]),

    //Treat the result as a JSON document

    ImportedJSON = Json.Document(GetElevations),

    //Navigate to the elevations data

    resourceSets = ImportedJSON[resourceSets],

    resourceSets1 = resourceSets{0},

    resources = resourceSets1[resources],

    resources1 = resources{0},

    elevations = resources1[elevations],

    //Turn the elevations data into a table

    TableFromList = Table.FromList(elevations, Splitter.SplitByNothing(), 

                                null, null, ExtraValues.Error),

    //Rename the column containing the elevations

    RenamedColumns = Table.RenameColumns(TableFromList,{{"Column1", "Elevations"}}),

    //Add a column containing the latitude

    InsertedCustom = Table.AddColumn(RenamedColumns, "Latitude", each Latitude),

    //Add an index column

    InsertedIndex = Table.AddIndexColumn(InsertedCustom,"Index"),

    //Use the index column to find the longitude for the current row

    InsertedCustom1 = Table.AddColumn(InsertedIndex, "Longitude", each Longitudes{[Index]}),

    //Remove the index column

    RemovedColumns = Table.RemoveColumns(InsertedCustom1,{"Index"}),

    //Set all columns to data type number

    ChangedType = Table.TransformColumnTypes(RemovedColumns,

                    {{"Elevations", type number}, {"Latitude", type number}, 

                    {"Longitude", type number}})

in

    ChangedType

 

It’s quite easy to edit the code so that it generates a list of latitudes and longitudes across the country of your choice…

Two things to point out:

  • To get this to work you need to insert a Bing Maps API key in the first step where indicated. If you don’t have one, you can get your own at https://www.bingmapsportal.com
  • The important step is GetElevations. The code is:

    Web.Contents(

    "http://dev.virtualearth.net/REST/v1/Elevation/List?key=" & BingMapsKey,

    [Content=Text.ToBinary(PostContents)]),

    You can see here how the list of locations is passed to the Web.Contents() function (documentation here) via the Content field; notice also that I’ve had to use Text.ToBinary() on the text that I’m passing in.

Here’s the output in Power Map:

image

You can download the sample workbook here.

Written by Chris Webb

April 19, 2014 at 9:00 am

Follow

Get every new post delivered to your Inbox.

Join 3,144 other followers