Watch videos with subtitles in your language, upload your videos, create your own subtitles! Click here to learn more on "how to Dotsub"

Messaging with Windows Azure Service Bus

0 (0 Likes / 0 Dislikes)
  • Embed Video

  • Embed normal player Copy to Clipboard
  • Embed a smaller player Copy to Clipboard
  • Advanced Embedding Options
  • Embed Video With Transcription

  • Embed with transcription beside video Copy to Clipboard
  • Embed with transcription below video Copy to Clipboard
  • Embed transcript

  • Embed transcript in:
    Copy to Clipboard
  • Invite a user to Dotsub
[TechEd 2013 - Messaging with Windows Azure Service Bus] [Ziv Rafalovich (@zivraf), Senior Program Manager, Windows Azure, Microsoft] Good morning. I am happy to see you all here. Before we start, did anyone attend the Integration Roadmap session earlier? Wow. So you should get a medal for the half a mile dash. I kind of realized that it's on the other side of the conference hall. So I have just 2 of the integration posters. Have you seen it? [inaudible audience member response] >>[Ziv] All right. You're sitting second row, and you are sitting fourth row. So for the rest, we printed 500 copies of this poster. I really recommend you to grab a copy or just download it from the Web. My name is Ziv Rafalovich. I'm a Senior Program Manager with the Windows Azure Service Bus team. This session will be focused on the messaging aspect of Windows Azure Service Bus. Before we actually start, I'd like you just to raise your hand if you've ever heard of Service Bus. Nice. I would say 80+%. And from those who have heard about Service Bus, how many actually developed applications using Service Bus? All right. We are in the 30s. And how many actually deployed an application using Service Bus? All right. Cool. We're getting there. Two years ago, just to find someone who had heard about Service Bus was an achievement. But I'm really proud to see this and see all of you here. So for those who haven't heard, those few, this will be a quick introduction. For those who have heard, I'll try to make this as a quick recap and then go and talk about what's new in Service Bus. We'll focus on a few of the many new features we have released during the last year. Then we will just spend some time talking about interoperability, interop, with Service Bus and then go and talk about the server. As mentioned yesterday in the keynote, Service Bus is part of the Azure Pack coming to System Center 2012 R2, Server 2012 R2. So we are there in this private cloud notion. Service Bus is a unified set of messaging capabilities. All share the same management and monitoring facilities. The relay is all about synchronous connectivity solution for those who need to project their on-premise line of business application as a public endpoint in the cloud to be accessed by cloud apps, partners, mobile devices, and so on. The brokered messages features include both queues and topics. Those are asynchronous message buffered solutions. Basically, you send a message to the broker and receive it later on. And push notification is a vital component in any modern mobile application. The Service Bus Notification Hub was recently released in preview and offers cross-platform scaled notification solution with advanced routing capabilities based on tags and it is usable from any back end application, either hosted in Windows Azure or on-prem. So with this mix and match of messaging and connectivity capabilities, basically we enable you to connect either on-prem or cloud components to other on-prem or cloud components. So to start with a story, basically this is a true story. Unfortunately, Friday I got an email from this customer that said, "Hey, guys. I'm sorry. We kind of reviewed this with you guys." "But you cannot share our name." So I have kind of changed the title and changed the industry, but the scenario is pretty straightforward. And this customer, let's call it MyLib, it's a library management organization who's been around for quite some time, and they used to have an on-prem facility, they had a full-blown application running on-premise, and they have embraced Windows Azure in order to create the Web presence. So you know libraries, they used to manage many locations, many branches. Think of it as like generic branches or locations or kiosks or whatever locations you have. They wanted to introduce a web app to serve their customers. So they have picked Windows Azure to develop their web app, and basically the web app was their portal, their MyLib.org website. And then they decided to keep using some of their back end systems such as the books catalog, their billing system, and their active directory for all of their workforce—the librarians and so on. Later on they realized the potential of Windows Azure, so they have also migrated the library management or the library engine application, as they call it. And this is a very common scenario. So we start with a customer who extends their presence in Windows Azure by developing some new application, usually web tier, and then, once realizing the full potential of the cloud, migrating their existing workloads or additional existing workloads to the cloud. So the nice thing about this customer is he decided and he really enjoyed using Service Bus. And he used Service Bus with multiple scenarios, so I'll try to take you through this journey. The first scenario is the hybrid connectivity with relay. His books catalog is a very complex application where they keep all the books and records and articles and e-books recently and so on. The good thing about their application is that it exposes a WCF endpoint, Windows Communication Framework endpoint, and all they basically had to do was to have this projected in the cloud using the Service Bus relay. So by changing the binding, basically using the Service Bus provided bindings for relay, they have created a public endpoint. And this public endpoint is accessed by their web catalog application, which is nothing but a wrapper which kind of transforms and creates nice HTML pages. Keep in mind when you're using Service Bus relay it enables location transparency. In other words, if I have my on-prem WCF service, all I need is to project it with a Service Bus relay. I don't need to poke any holes in my firewall if I use NAT. Service Bus provides NAT traversal. We like to call it a turnkey cloud connect, basically just change one line or one configuration. We've seen it with customers. They had to evaluate for several weeks, but the actual work was done in 2 hours. It provides scale out. So multiple servers—in this case they only have one instance— but multiple instances, up to 25 instances of my line of business application, could use the same public endpoint. And by going that way, Service Bus actually serves as a load balancer as well. This endpoint is secured using the Windows Azure Active Directory, a.k.a. ACS, and basically this is how MyLib used relay. They also used relay for Active Directory. They've written a simple WCF wrapper to validate the librarians' login to the application. I did not call this out because since the time they've developed this solution we have already announced the Windows Azure Active Directory. So now they are rethinking whether to embrace Windows Azure Active Directory as well. The second scenario was to leverage their billing system. Billing is always kind of sensitive, especially when you deal with credit cards and sensitive information. So what they've done is they decided to leverage Service Bus queues to decouple their billing system and have it still on-prem while you can either submit purchase orders from their website as well as from the library. So when you type your credit card information in their website or just swipe the card at the library, basically a message is being posted, sent to the Service Bus queue, later on being picked by the billing system. Again, as I mentioned, this is an asynchronous way, so the client application does not need to wait for the complete process to be successful or fail. MyLib trusts their users, especially since libraries are usually low amounts. But using Service Bus queues you can decouple your components, decouple your application by having a queue in between a web tier or a line of business or any 2 work roles or any 2 application components. You get load leveling by having the receiver receiving messages at its own pace. In other words, first week of January all the users need to renew their subscription and they swipe their credit cards or type their credit card numbers, but the billing system basically has one instance. So this instance kind of will process messages with its own pace and slower. So this is the load leveling. Scale out. Basically, they could scale their application by adding more instances. Those instances are what we call competing receivers. So if 2 instances try to receive messages from the same queue, Service Bus guarantees that a message will be processed by only one receiver. We do this by enabling 2 receive methods. One is receive and delete, and the other is receive and complete. With receive and delete, once you receive the message it is no longer there in Service Bus. That's good. That's simple. But if you failed while processing the message, you have lost the message. In receive and complete, once you receive the message we kind of lock the message, and you need to explicitly complete the message when you're done. In this way, the message is locked for the duration of the process, and other competing receivers won't be able to receive the message. So once deploying both the library's component and their website to the cloud, they realized that they have new business opportunities, such as book reservations. And the idea is they have many locations, and the catalog is different in multiple locations, so the idea is that you can reserve a book and they will make sure that this book is shipped to the library next to you. So what they've done is they've leveraged the pub-sub with topics. Basically, when you reserve a book, a message is being put to the Service Bus topic, which has both the source library ID and the destination. And those 2 libraries will get the message using the pub-sub capabilities of Service Bus. So the source basically will take this book and put it in the shelf for the UPS folks to pick it up, and the destination will wait for this book, and, once there, make it in the reserved shelf. Service Bus topics and subscriptions offers pub-sub capabilities. With those capabilities you can have broadcasting. Broadcast basically means that a message which is sent to the topic is being received by all subscriptions. In addition, you can define filters for each subscription, basically filters just to have subset of the messages which are being sent to the topic. In the scenario I just described, each library had its own subscription where the filter said, "If my library ID appears either in the source or the destination, I would like to have this message." You can also create partitions based on ZIP code or address or whatever your business needs are. In addition to that, pub-sub enables content-based routing so that the sender does not need to know even which are the receivers. Basically, the receivers specify their own filter in the subscription and the sender just sends the message. And on top of the pub-sub we have documented and implemented multiple messaging enterprise integration patterns. Last year's TechEd session was dedicated to those. So scatter-gather, resequencer, splitters, and all of those could be implemented with topics and subscriptions, and if you'd like to learn more, just either browse for last year's TechEd or just ping me. [New Features] That was a quick introduction. For those who have been with us for some time now, I hope you realize that we keep ourselves busy. We have basically released 3 major releases in the last 8 months, and it started last year, last October with the SDK 1.8. By this time, we have released many, many new features. I'd just like to focus on 2 of those. The first was the message lock renewal. As I mentioned, in receive and complete there is a timeout for you to process the message. This timeout, the default value is 1 minute; the maximum is 5. However, we've seen customers who need more time to process messages and ask us to extend this timeout. So with the message lock renewal, basically what we enable is for a customer to renew this lock every 5 minutes until they are done with their processing. And the idea is to keep the timeout small enough that if the component processing the message fails, others will be able to pick up the message in a reasonable time. The second feature we have introduced was forward messages between entities. The idea is to tie or connect—we call it forward to— multiple entities. This came after a request from a customer who had many, many branches and they basically needed 1600 subscriptions to a topic. And although a topic can support up to 2000 subscriptions, throughput and performance decrease when you reach the several hundreds. So with the forward to feature they were able to connect subscriptions from the source topic to multiple topics and basically create a spanning tree of topics. So you have 1 topic forward to 10 or 100 topics, each forwards to 10 or 100 topics, and by doing so, you can easily achieve thousands of subscriptions with reasonable, accepted performance. Then earlier this year, in July, we have announced the preview of the Notification Hub. I will not get into details. If you'd like to see demos and learn more, Abhishek Agrawal will have a session Thursday morning about connected clients with Service Bus where he will demonstrate the Notification Hub. It is still in preview, and we're looking to release it for general availability later this year. Two months ago we have released SDK 2.0. In this release we have introduced new ways to authenticate clients. A feature called auto-delete idle entities, this is more of a cleanup feature. So if you have a topic and no one was using this topic for quite some time, we will clean it up for you. It applies both to queue, topic, and subscription. So the idea is that if you scale your application and each instance of a work role creates their own subscription, then you basically have to look into cleanup. And with this feature, you specify the time duration for this idle cleanup. So if I just started a worker instance, I created a new subscription, I specify 7 days for idle timeout, and once I take out this instance, after 7 days the subscription will be deleted. It is also useful for test environments. We've seen customers starting many, many new queues and topics and reaching the 10,000 queues limit. And then when we approach them and ask, "Hey guys, are you really using 10,000 topics?" they say, "Well, no, that's really our test automation." And they said, "Now we need to run advanced queries to say which queues to delete and when," and so on and so forth. With this feature, all they need to do is specify the timeout, and usually the tests are short duration, so they are done in 2 hours. So within one day they can clean up their entities. With regards to the rest of the features, we'll just have a deep dive soon. We've been hearing this request for an event-based programming model. We've seen customers struggling with writing efficient and effective receive loops. We've seen partners and and MVPs releasing frameworks to do this receive loop and have basically an OnMessage API. So with the SDK 2.0 we have introduced this event-based programming model. What this actually means is that you specify the callback— what to do when the message arrives on your queue or subscription. Let's drill into a quick demo, and we will get back to this demo later on today. This is my application, and what I do is I send 10 messages in a loop and then I specify the options for the OnMessage. Basically what I'm saying is, "Please call complete for me." "So if my event handler did not throw any exception, please do the completion for me." Then how many threads to run concurrently? In this case I'll only have one thread or one concurrent pending call. And by the way, this receive loop is all using async APIs, so it is way more efficient than calling the synchronous APIs. And callback to catch exceptions. If I throw an exception from my event processing API, this LogErrors function will be called. This is the receive loop. OnMessage call this function, ProcessMessage, with the options specified above. And surprisingly, the ProcessMessage will just trace the content. This demo basically will send 10 messages to a queue and receive those 10 messages later on with this implicit receive loop. Very, very simple. I just start it and it connects to the cloud, and sending 10 messages, receiving 10 messages. It's very, very simple and straightforward. I'd like to highlight a new capability we have released in SDK 1.8, which is to specify the connection string. As you can see in my config file, I have added an entry called ServiceBus.ConnectionString which is populated with the endpoint value I took from the Windows Azure Management Portal. This basically replaced the API and helper classes we have used before this release, and it's a very rich feature where you can define all kinds of changes and tweaks to the connection string. That was the basic demo of OnMessage. Next we have introduced a new way to authenticate clients. What we've heard is that with the Windows Azure Active Directory, ACS, it's a great solution to federate identities. However, many customers do not really require federated identities, they just need the password. So with SDK 2.0 we have introduced the Shared Access Signature authentication. Basically, it is available both in the Windows Azure Portal as of yesterday— I will just give you a quick look at the new extension to the portal— but it is also available as an API. It is available at the level of a namespace as well as the level of entity. And by entity I refer to queues and topics, Notification Hubs, and relay. Support for different SAS rules for subscriptions will come soon. So at the level of a namespace you can either use the portal or our REST API. And at the level of entity you can use our .NET SDK, and this is how you basically create a SAS rule for an entity. As part of the entity description, being a queue description in this case, I just add a Shared Access Authorization rule specifying the rule name, the key—and I'm just generating a random key— and the access rights, access rights being either manage, send, or receive or all of them together. And then when I create a queue I just provide the queue description. I can update the queue and modify those authorization rules. Reaching out to this queue, connecting to this queue, you can use the Service Bus environment and create the message factory explicitly or use the connection string I just showed earlier. So what I'm going to do now is I will switch to the portal, create a new SAS rule for my namespace, and use this as a connection string to the demo we've just seen. I am switching to the Windows Azure Portal, and this is the Service Bus namespace I'm going to use today. This is new, released yesterday. Basically, I can see the shared access policies for this namespace. Those are the namespace level policies. I can create a new policy called demoRule, and this demo rule will include management, send, and listen. And I do save. Successfully saved. Blah, blah, blah, blah, blah. And when I go and look at the connection info— this is again new extension to the portal— I see beside the existing old ACS-based connection string new SAS connection strings. And this is the new rule I've just created. So I'm copying this one, and then I switch to my config file and comment out the old one and paste a new connection string. If you just compare the two, you will see that this one has a shared access key and this one had an issuer and a shared value. So with this new SAS— Oh. "Are you still debugging?" Yes. So with this new SAS rule— There is an extra space. Yes. All right. Now using the SAS, it's the same application. And just to show you that it is really based on SAS, what we'll do is we'll go and delete this rule from the portal. Let me just exit the application. So now we will just go and delete. It was the demoRule. I'm going to delete this one, save, and then I expect now to get some sort of unapproved or authorization or 401. I'm getting an unauthorized exception. So I was using the SAS rule. And we'll get to this demo later on. [Demo - Shared Access Secret (SAS)] Okay. Back in SDK 1.8 we have also released the entity query API. The challenge we tried to approach here was how to manage multiple entities. And by multiple I mean tens or hundreds or thousands of either queues or subscriptions or topics. And we have decided to go and invest into those query APIs. Those are available both in .NET or REST, and the idea is to get a list based on a filter. The filter is defined in an all data format, so just 3 samples. You can get all the queues which have more than 10 messages. This is really valuable information. You can build your own dashboard, your own monitoring tool, based on those queries. Get me all topics, which starts with the name foo. This is a string manipulation on the topic's path. And they were accessed less than, or prior to, some given time. So I can go and have a look at all of those queues which are not active. And also in 1.8 we have introduced one of the attributes called AccessedAt. For each entity, basically we have created time, update time, and access time or CreateAt, UpdateAt, and AccessedAt. Creation is creation, update is all the management operations, but then we have introduced the accessed time, and accessed meaning that any runtime operation hitting this entity being queue, topic, or subscription— in other words, if a message was sent, received. If someone tried to receive a message and there wasn't any message, this still will stamp this access time. And we have also aggregated or, in a way, federated the access time of subscription to the level of a topic. In other words, if I have a topic with 100 subscriptions the last access time of the topic will be the max last access time from all of the subscriptions. So it's a pretty effective way to monitor your entities in your namespace. The last sample is basically at the level of subscriptions. Subscriptions are defined within the scope of topic. So you cannot query for subscriptions. In your namespace you just query for subscriptions within your topic. So you need to provide the topic name. Basically, this sample looks for all the subscriptions defined in this topic which have more than 10 messages and were accessed prior to this provided start time. This is very effective, again, if you have the library. I want to monitor and see which libraries do not pick up their messages. So I can use these sort of APIs. Yes, please. [inaudible audience member question] [Ziv R.] I'm sorry. The question was? [inaudible audience member question] [Ziv R.] We support only this all data filter as strings. The question was whether there is a link support. Not yet. By the way, this API is available in REST as well, so this is more or less how you will make a REST call to this API. [inaudible audience member question] >>[Ziv R.] Sorry? [inaudible audience member question] >>[Ziv R.] No, it's not peek. We are not touching the messages; we're just running on the— We'll get to peek in 2 minutes. That's a great question because the next feature is peek. But the idea is to browse the metadata of Service Bus but reach out to the real-time counters of message count or last access time. So these are pretty powerful APIs to use. Thank you for the question. We've heard customers looking to get more visibility into messages in the queue. Besides counters, they wanted to see what are the messages. And this is mainly for debug scenarios, but there are some other workarounds. This feature could be used as workaround for other advanced scenarios. It's quite useful. The idea is we provide a peek API. As I mentioned, when you do receive on a message, it's either deleted or locked depending on the receive mode. Think of it as a new receive mode which is just peek and don't lock. This is why we call it peek. Or receive without lock. With the peek you can either get the first message in the queue and iterate the queue, or you can ask for messages starting from a sequence number, or you can ask for messages in a batch. "Give me the first 10, the first 20," and so on. We have released a pretty straightforward, simple sample for this API. Basically, it prints the content of your entities. Great for debugging. Yes, please. [inaudible audience member question] [Ziv R.] This will upgrade the AccessedAt? No. No. No. Thank you. Great question. The question was whether this call for peek will update the access time. The answer is no. For those of you who use the .NET async API, we have introduced the support for await. Those Service Bus APIs have always been both synchronous and asynchronous. Instead of calling begin receive and end receive and catching the result, you can use await today just like you do with other .NET APIs. [Service Bus support with AMQP 1.0] Interop. How many of you have heard of AMQP? Ah, quite a few. All right. One of the promises of integration in general, one of the key aspects is the ability to connect multiple applications from multiple platforms, from multiple operating systems. However, over the years we've seen— messaging has been around for 20+ years at least, maybe even more— but traditionally, we have seen vendors releasing their own proprietary protocols, supporting with their own proprietary set of client libraries SDKs. Basically, this is great job security for the vendors because you cannot really port, you cannot really migrate. If you want to migrate, if you had enough with your existing vendors, you need to actually touch each and every component in your application. And sometimes there are parity issues and so on, you need to find different libraries. That's a challenge. In addition to that, it's hard to integrate components running from multiple platforms. We have seen many vendors offering bridges, like bridging 2 protocols or 2 applications or 2 operating systems. We've seen many vendors offering bridges. And then as new platforms are becoming pretty common, like NoJS, we have seen vendors struggling to provide more and more client libraries. They need to keep up with the new platforms. They find it hard to embrace any open source technologies and so on. So all of those led to the definition of AMQP, Advanced Message Queuing Protocol. It was an initiative by JPMorgan Chase, a financial service user firm rather than a technology firm, who realized the challenges. And once they've created this initial version of their AMQP, they have brought on additional 20+ companies, including technology firms, and we, Microsoft, have been fully engaged with this effort. The idea was to enable cross-platform applications built using different brokers, using different libraries and different frameworks from different vendors to operate together, to interoperate. The AMQP is an efficient protocol. AMQP defines the binary wire-level representation of messages. It is reliable, offering a range of reliability methods, starting from fire-and-forget and all the way to exactly-once delivery. Portable data representation means that a .NET dictionary object will be translated to a Java map, and this is because those 2 platforms are translated to the protocol itself. It is brokered independently. In other words, it does not introduce any requirements on brokers, making the adoption by existing brokers really straightforward, having the low entry barrier. And it is also flexible—supports multiple configurations, either peer-to-peer or client-to-broker, or broker-to-broker. Two weeks ago we were really proud to announce that Service Bus supports AMQP 1.0 General Availability. It has been out as a preview for several months, and we are finally there in the cloud. This support is available using the new SDK, SDK 2.1. Besides supporting the AMQP protocol with our Service Bus, we have also released this SDK 2.1, which is AMQP library so you can use it both for our proprietary protocol as well as AMQP. Besides our library, which is a .NET library, other vendors released support for AMQP 1.0. Those are just the client libraries. The Apache Qpid project is an Apache project which offers several platform support. The good thing is with the Java. The Java client libraries basically are JMS implementations, Java Messaging Service implementation. In the Java world, JMS is just an API defining how messaging should behave. With AMQP and Service Bus support for AMQP you can simply use your Java JMS application, go pick up one of those 2 AMQP client libraries, and start talking to Service Bus and have any other platform on the other side. Though we have shown this demo before, what I'll do, I'll just show a subset of this demo. I'll have a .NET component sending chat, chat application sending messages to a topic, then I'll have 2 subscriptions. One is a .NET in Windows and one will be Java. I have 2 programs here, the sender and the receiver. The sender is a very simple chat window. It looks whether the topic exists, and if the topic is not there, it creates a topic with 2 subscriptions— one is for the .NET, one is for AMQP— creates a client to this topic, and in a loop reads a line from console and sends it as a message. The receiver is even simpler and should be familiar to you by now. I'm using the OnMessage and just type the message. It's a simple chat. And just to compare, that would have been a very simplistic and wrong way to write a receive loop. So just to compare the two and going back to the first demo we have presented, there are many pieces missing in this while loop— what to do with exceptions. It's using synchronous receive, not asynchronous, so on and so forth. So by just calling the OnMessage, the code is better, performs better, and is simpler. So what we'll do is I'm going to start the sender and I'm going to start the receiver. Hello. Hello. You see 1, 2, 3, 4, 5, 6. It just worked. You may say, "Nothing new here," right? Well, almost, yes. The only change that you would expect to find here is this tiny extension to the connection string, TransportType=Amqp. This is the only change we require from the client side to talk AMQP to Service Bus. And yes, someone asked me yesterday, you can send messages using AMQP and receive those with the proprietary Service Bus messaging protocol. Yes, please. [inaudible audience member question] [Ziv R.] Am I parsing the connection string? The connection string is this, and our client code basically— [audience member] I mean the key. [inaudible] [Ziv R.] This key came from the portal. [multiple audience members speaking at once - inaudible] [Ziv R.] Ah! Okay. The question was about this key name. Basically, the answer is if you implicitly— Where is the connect? If you implicitly call namespace create, then we are looking for that key. What you can do is you can do something like this: CreateFrom— Let's stop the application. CreateFromConnectionString and provide whatever you want. Are you there? I need to start the receiver. And you see the message was waiting. That was just the .NET part. Nothing new here. Now let's switch to my eclipse environment, and this is the Java component, Java piece. What do we do here? We start by loading the Apache Qpid AMQP 1.0 JMS implementation. That's the client library. Then we create a connection factory based on this SBCF. So following your question, SBCF will be defined in my config file. This is my Java JMS connection string. With this factory we create a topic. We look up for the topic using the entity name from the configuration file. We create a connection and then create session on that connection and then create a topic subscriber connecting to Service Bus subscription. So you can see that while the terminology is kind of similar to Service Bus, it is different. This is JMS terminology. And Java JMS developers should be familiar with that. Basically, my class also implements the callback for MessageListener. This means that I need to implement the OnMessage function call. And, surprise, surprise, when I receive a message all I'm going to do is I'm going to put it on the console. So running this sample— Before going there, I want to show you the configuration. This is how an AMQP connection string looks for JMS and other platforms. So in the AMQP documentation in MSDN you will find how to convert the Service Bus connection string to an AMQP connection string for non .NET platforms. You see, it's kind of an owner, password, and then @ an address. So it's a bit of a different format. Today we don't really offer this in the Windows Azure Portal. You need to do some manipulation. It's all captured in our developer's guide in MSDN. So what I'll do is I'll start running this Java up, and what I'm expecting to see, if it runs— All right. Finally. So you can see the Java console, all of my messages, and just to show you that it is live, then Here we are again. You see it from the Java, and you see it from the .NET, just to show you that it actually runs. What I have is I have a .NET sender using Service Bus library, .NET receiver from one subscription, and then a Java receiver from another subscription all talking AMQP. Okay? Interop with Service Bus. [applause] Thank you. This is the result of, I would say, 3 years of effort. [laughter] And I'd like to thank David Ingham, who is an active member in the AMQP community. Thank you, David. We have 25 more minutes left, and now I'll switch gears to Windows Server. [Service Bus for Windows Server] Service Bus for Windows Server. We kind of never called it actually Service Bus Server, although if you look at the forum, it's all about SBWS, people start calling it. So Service Bus for Windows Server. That's the official name. When we thought of why to bring Service Bus on-prem, basically we had one customer. Luckily, this customer was big enough. It was SharePoint. SharePoint Office 15, their workflow. SharePoint team decided to take dependency on the workflow team, and the workflow team releasing Workflow Manager decided to take dependency on Service Bus. So we are, in a way, the state machine for Workflow Manager, which is the workflow engine for SharePoint, both on-prem and in the cloud. So as part of this deal, we had to release Service Bus as a server product by the same time, same ship as Office 15. So last October we had released the Service Bus for Windows Server 1.0. And besides SharePoint, we looked for other scenarios where Service Bus could be valuable, and we came back to the messaging middleware, the MOM, the message-oriented middleware. We had enterprises who developed their homegrown ESB or homegrown messaging platform, taking dependencies on all sorts of technologies, either Microsoft or non-Microsoft, and then they asked what was simple. "Provide us with a pub-sub capability, something like extend MSMQ." And by the way, we are still maintaining MSMQ. It's the same team. We are the same team. "Provide us pub-sub on-prem with scale which is highly available, which is manageable, and give us all this goodness that you bring to the cloud." That was the first type of customers. And then we've heard another thing from the Azure developers, who asked for an emulator. So several of the Azure components have this on-prem emulator in Visual Studio, and we decided that we don't really want to have an emulator. We want the real thing. So for those we kind of took the server and made it a dev box. By dev box I mean it runs on client OS, it can use SQL Express. You can even tweak some of the runtime settings to relax timeouts, the peek lock and so on. And by the way, those features are not for production. So if you plan to use production, please, you have to use Windows Server and SQL, either Standard or Enterprise. So for production we don't support client OS and SQL Express. But as a dev box it just works, and actually, I have Service Bus installed on my Windows 8 laptop. The third was the most interesting customer segments. Those were ISVs or enterprises developing softwares and they sell this software. Basically, their request was simple. "I develop once. I want to sell it in 2 flavors, in 2 skews." "One for large customers as a DVD to install on-prem, "full-blown server product with all the enterprise-ready management and scale and so on, security." "And on the other hand, I'd like to leverage the cloud, Windows Azure, "and deploy my service as a PaaS or as a SAS offering supporting multi-tenancy, and I want to write my application only once." So for those what we call flexible deployment, for those we kind of committed full symmetry, at least for messaging features. And for those customers we have introduced connection strings for both server and service. So all they need to do is just to change this connection string and run their applications against or with Service Bus for Windows Server. Yesterday in the keynote, Brad Anderson was talking about the Azure Pack. The Azure Pack— Yes. Question. [inaudible audience member question] [Ziv R.] The question was about Service Bus and the BizTalk cloud. I guess you did not attend the previous session, the Integration Roadmap. But the idea is that Service Bus is technology as a platform while BizTalk is a product, is a full-blown product which does way more. BizTalk Services uses Service Bus as an endpoint, either for incoming or outgoing. BizTalk Server in 2013 have introduced out-of-the-box support for Service Bus for messages. And with their WCF extensions we have also released support for all the versions of BizTalk. And there are a few blog posts. If you are interested in that, I can point you to how to connect Service Bus to or from BizTalk 2010. In other words, BizTalk is a product; Service Bus is a platform. Service Bus just does messaging and routing while BizTalk does the transformation, translation, BAM, you name it, business rules and so on and so forth. By the way, if you are interested in this, let me just do a bit of PR. We are going to have a Birds of Feather session tomorrow noon, I think, titled Lightweight Integration. It will be me and a guru from the BizTalk team, and we will talk about those BizTalk Service Bus better together, build versus buy, and map those 2 integration scenarios. Thank you for the question. As part of the Azure Pack we have brought Service Bus to this private cloud initiative. And what does it mean? The idea is to manage Service Bus on-prem for large enterprises and hosters just like we do in Windows Azure, introducing tenant portals so that tenants can go and provision their own namespace and their own entities in the portal; also introduce an admin portal where admin can either provision new Service Bus clouds, as they call it, or basic management capabilities for existing Service Bus clouds. So think of a hoster scenario. Each tenant or even an enterprise, each division requires their own Service Bus server. So while we have security isolation within Service Bus, we've seen customers asking for hardware isolation. And with Service Bus 1.1 Preview, basically we enable you to manage multiple Service Bus forms or servers or clouds, as they call it now in the portal, from a single portal. So you connect to multiple Service Bus clouds and you specify plans as an admin. The plan may be public, may be private. And then when users come to the portal, they sign up for subscriptions. If they sign to a subscription which has the Service Bus as part of the available resources, they'll get Service Bus next to virtual machines, websites, SQL. So Service Bus is kind of in a really good list of big Microsoft technologies such as VM, SQL, and so on. But the intention was that it might be that we are the first of a wave of bringing back Azure services back on-prem. And besides the private cloud or the Azure Pack, we have also brought all the latest features. So basically, if Service Bus 1.0 supported SDK 1.8 last October, the new preview or the new release will support the current SDK. So everything I've just discussed so far—SAS, RenewLock, entity queries, the event model—is all coming to Service Bus Server. Word on platform. We were kind of talking about the runtime capabilities— topics, subscriptions, authentication, so on— but when we decided to release the server, we had to realize that there are some more requirements on the platform, because in the cloud we do everything. We provision, we manage, we monitor. And believe me, if you follow our tweets, we have this notorious thing we called servicing loop, which is each one of us serves as basically the one on call for one week every 2-3 months, and we hear, we see the pain, we see all the crashes, and I forgot to mention as part of this Service Bus telemetry we even introduced the KPIs. So we will count the number of exceptions each customer gets. And by the way, you can get those in the Azure Portal today as part of the telemetry. So we see customer exceptions, we see exceptions from the service. We have our own runners to monitor the service, and we realized that we need to simplify this for the server admin. So the first step was to make a bet on the best components Microsoft has to offer. This means the Windows Server and SQL Server. So for Server we support 2008 R2 SP1 and above, and, yes, we will run on the next Service Bus Server 2012 R2. I try not to say the color. We already run on the latest server bits. SQL as well, starting from 2008 R2 SP1 as well as the latest bits. Highly available. We brought the high availability concept, and we run a ring of nodes, and when one is identified as down, then we redirect all the load to the other nodes. Since we brought the cloud capabilities, we require 3 nodes for high availability because our internal mechanism is quorum based, so we require 3. So HA equals 3 nodes in Service Bus. We have brought Windows authentication. So for enterprises, that's basically how they do authentication. No ACS, no shared secrets. You have AD, you can specify AD group or domain users. We have also released SystemCenter Management Pack set of administrator PowerShells and the portals which I will show briefly. For ISVs, besides SDK symmetry, we have also brought the SAS authentication. So ISVs basically do not require domain join or AD at all. So we have brought the SAS capabilities. And as I mentioned for developers, the one box topology, which is kind of like an emulator but it's the real thing. [Demo - Service Bus for Windows Server 1.1 (preview)] Before going to the demo, I just want to recap the year we had. Just adding the server releases, you can see the release cadence. What I like with this slide is the fact that there are multiple releases in the cloud, 2, 3, 4 a year, but what we try to do is we try to keep up a single annual release for Service Bus Server for Windows Server. Last year it was aligned with the office release; this year it is aligned with the Windows Server SystemCenter. So we are part of a larger wave. Yes, please. [inaudible audience member question] [Ziv R.] The question, let me repeat it and see that I understand. What components are we using SQL Server, and what are we using SQL Server for? Service Bus, both in the cloud and on-prem, we are reliable messaging, so each message we store in a database. So basically, we're just using the SQL databases, both SQL Azure and SQL Server, Standard or Enterprise. If you have Enterprise, you can leverage the high availability options of SQL. We don't provide high availability on top of SQL. So if you still need to back up your databases and use SQL AlwaysOn to make your databases highly available, we will try to make our compute nodes or our Service Bus workloads highly available. In the cloud we use SQL Azure. I forgot to mention it, but you can also install Service Bus for Windows Server on the Windows Azure IaaS virtual machines, and then you have an option to use either SQL on VM or use SQL Azure. So in other words, you can go hybrid and put your Service Bus Server on a VM and pick up SQL Azure. Performance varies, depends on your scenario, but it is supported. Again, to recap, the release cadence is basically whenever we release a server, even in preview, it will have its own matching SDK. So if you are using Service Bus 1.0, you can use the SDK 1.8 but not the SDK 2.0. All right? And some features such as Notification Hubs and relay are not available yet on-prem. Before going to the demo, I'll just do a quick disclaimer. It worked for me earlier this morning from the hotel. Here I don't know why but I cannot ping my machine. So we'll try to go there and see how far we can get. This looks like a Windows Azure Portal, but it's not really. This is the admin portal of the Azure Pack. And though yesterday in the keynote you saw a full configured, full populated private cloud, here in my example, which I ran from a VM back in my office, I have 4 Service Bus Servers connected, and those 4 Service Bus clouds, we had a bug bash with the team, so I gave my portal. "Hey guys, let's see what it means to manage multiple Service Bus forms." So this is the admin experience to manage multiple Service Bus clouds or forms. And once you connect to a Service Bus, you define plans. And plans is like your cell provider. You sign up for a plan. So in each plan you specify. Think of it as like a menu. What's available? You see that in this plan, only Service Bus is available from this cloud as B demo. This is the admin experience for managing multiple Service Bus clouds from one place. The talent experience, let's try to do a demo and see how far we can get. Now I am signing up. This looks like Windows Azure, right? But you can see that it runs from my dev box. So I'm going to sign up with [email protected], my favorite password, which I always forget. Let's see how far we can get. All right. Validated. So now I've created a new user in my private cloud. And by the way, the bits I'm currently using are from, I think, Saturday morning, so it's kind of really— [sighs] We'll see. There is my Welcome screen. I'll just keep it. Now I need to sign up for a plan or for a subscription. So what I'll do is I'll take the free plan. I always like to take free stuff. Process my request and failed, but that's a known bug which guys should fix tomorrow. I just refresh the portal, and I should be able to see Service Bus available. My subscription already created. I see my subscription is there. So again, this is the private cloud tenant experience. And now I can create a Service Bus demo namespace. I create a Service Bus namespace in my private cloud, and I am a tenant, I am not an admin. I have no admin privileges on this Service Bus form itself, and you can see that the portal is actually the same. I can create a queue. All right. And you see the queue and so on. Let's try to do the final trick. What I'll do is I'll try to use this Service Bus namespace on-prem with the application we've just seen an hour ago. So I go and look for the connection string, and there is my SAS rule. Each namespace is created with a default SAS rule. I copy it. Going back to the sample. Not this one. Just going back to the config file and replacing the endpoint with the one I've just pasted. And you can see the endpoint now. It's a pretty complex string. So what I see is that it will go to this machine. We have also introduced an STS endpoint. The STS is located on the same machine, and the ports, and this is the shared access that I mentioned before. And now the final trick. What's your bet? Will it work or not? We'll see. It worked earlier today. It did not work an hour ago. Let's see how far we get. And it did not work. All right. The problem is that for some reason, when I ping my laptop from the portal, it does not find it. Something with the networks went on. I probably need to flush some DNS entries. But you got the point, right? Consistent, symmetric experience from the public cloud to a private cloud. You can still use Service Bus, just the runtime capabilities without the portal, just as you used before. It will be available as a preview later on this month. I think we are right on time to take some questions. Yes, please. [audience member] A question about VLANs. Is there a support for on-premise endpoint [inaudible] [Ziv R.] The question was about support for additional APIs besides WCF. And the answer is yes, there is a plan. It's ongoing. We will announce it, I think, later this year. Not at the moment. And just before you leave, I'd like to point you to related sessions where Service Bus will be discussed. I mentioned tomorrow morning the Birds of Feather. Then Thursday at 10:15 the B336. This will be the other side of Service Bus— all the clients, Notifications Hub, and so on and so forth. The other sessions, the 405 and the 340, are mainly BizTalk related, but in those sessions they show hybrid solutions and extensions of the BizTalk Suite, both server and service with Service Bus. Any other questions? Yes, please. [audience member] Is there some kind of support for dead messages? [Ziv R.] Support for? >>[audience member] Dead messages. [Ziv R.] Yes. Support for dead messages. We call it dead letter queue, not dead message. When you create a Service Bus queue or topic, you specify whether to enable the dead letter queue and when a message will be expired or thrown by the client. Those will be piling up in the dead letter queue. Yes. Yes, sir. [inaudible audience member question] >>[Ziv R.] Yes. The question was the STS is used for AD authentication. Yes. [inaudible audience member question] [Ziv R.] No. You can use just AD. It will use by default the same domain as the machine is joined to. Yes, sir. [inaudible audience member question] [Ziv R.] If you call a receive and complete, you need to explicitly specify whether to complete or to defer or to abandon the message. If you use the OnMessage, there was one of the settings, Auto Completion. Let me just switch to the code just for one sec. So you see this one, this setting AutoComplete? This means that you don't need to call complete. We will call it if your handler completed successfully. Okay? You can still overload and call defer or abandon to the message. Yes, sir. >>[inaudible audience member question] [Ziv R.] Does that work with net messaging binding? Great question. I am not sure. Please take it with me and I will point you to the— Actually, do you know? Does this work with— No? [inaudible audience member response] >>[Ziv R.] All right. Just approach us and we'll answer it. Yes. [inaudible audience member question] [Ziv R.] For the dev box? >>[inaudible audience member response] [Ziv R.] Windows 7, 64 bits and above. Sixty-four bits only, not 32. Yes, sir. >>[inaudible audience member question] [Ziv R.] Any features not supported on server and in the cloud? Not yet for messaging. Basically, we are deciding on a date. We branch the code and then stabilize server. So there may be some new features coming in the next month or so but not that I am aware of. Yes, sir. [inaudible audience member question] [Ziv R.] The question was how far or what's the release cadence for Server? We would like to think of it as an annual cadence, especially if we are now aligned with the Windows Server wave Dave mentioned— that they will move to this annual release cadence. We just proved that the second release came within a year or it will come within a year. We plan to release it later on this year, fall or late summer, as they call it. My hope is to have an annual release because we develop— The mentality of the team is develop cloud first. So we always develop to the cloud, and the good thing is that we have a chance to make sure those features mature enough in the cloud for us to bring them to Server. For example, last year with Service Bus 1.0 there were tiny features which were not mature enough. They came last minute. So we decided to keep them out, kind of really small features. Any other questions? So with this, feel free to approach us. Thank you very much, and see you later on this week. [applause]

Video Details

Duration: 1 hour, 15 minutes and 52 seconds
Country: United States
Language: English
Genre: None
Views: 5
Posted by: asoboleva99 on Jul 9, 2013

http://channel9.msdn.com/Events/TechEd/NorthAmerica/2013/WAD-B310#fbid=kG7OLm6xV3l
Langauges to MT: Fra, Ger, Spa, Bra, Kor, Jpn, Rus, CHT, Ita

Caption and Translate

    Sign In/Register for Dotsub to translate this video.