2007.12.11

UpdatePanel UTF-8 encoding woes

Kategori: Programmering — Michael Schøler, Kl. 19:06:00

Encoding frustrations...
The last couple of days I have been struggling with text encoding issues in a C# web project, and in the process I have found an “official Microsoft hack” that might be of interest to others as well - hence this article. Using hacks to solve a problem is not best practice and should for all sakes be avoided but alas it is the only solution if you find yourself stuck to developing your web project in Microsoft Visual Studio 2005 so read on in that spirit. Note: Microsoft has fixed the bug and expects us all to upgrade to Visual Studio 2008 - they won’t fix 2005. Sure, everybody can change their development environment from day to day.

I was pulling hair out at an exponentially escalating rate, almost rendering me bald, trying to figure out what the “#!” happened when submitting text values from a standard C# aspx form. The web application I was working on had due to project specifications been forced to use ISO-8859-1 encoding by setting the globalization property in the web.config file and by content-type specifications both in the page declaration and meta tag information in the page’s head section but texts were in a seemingly sporadic way arriving at the server as default UTF-8 encoded text?!

By following the traffic between the server and the client browser, using the Fiddler2 debugging proxy, I could see that text values got transformed from ISO-8859-1 encoding (standard latin-1 charset) into UTF-8 (Unicode Transformation Format) in the POST request. That suggested the client browser was responsible for the errornous encoding. So, I tried IE6, IE7, FireFox, Opera - and they all failed. Clearly the problem had to be somewhere else than in the browser - it is simply not possible for Microsoft, Mozilla and Opera to be in such uniform agreement.

Googling the issue led down many blind paths, as it was unclear exactly what was the cause of the error, and it was therefore difficult to figure out the “magic” search terms to use.

So, I ended up meticulously trimming down the webpage one tag and item at a time - and finally I had managed to locate the bug: Microsoft. Now all that remains is to locate the responsible employee and then activate a Microsoft W.S.Y.P.tm punishment of an adequate magnitude.

The Microsoft ASP.NET AJAX Extensions framework wrongly treats all text strings as UTF-8, regardless of the specified encoding of the web application. This means that by using partial updates in an UpdatePanel the ajax framework gets busy talking jibberish to your web application server and you end up spending countless hours wondering about what the heck goes on.

You can try the following simple code example to see it for yourself (ASP .NET AJAX enabled C# website project), or skip it and read on.

Enter this in your web.config file in the <system.web> section:

<globalization requestEncoding=”iso-8859-1″ responseEncoding=”iso-8859-1″/>

TestEncoding.aspx

  1. <%@ Page Language=“C#” AutoEventWireup=“true” CodeFile=“TestEncoding.aspx.cs” ValidateRequest=“false” ContentType=“text/html; charset=iso-8859-1″ Inherits=“TestEncoding” %>
  2. <!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd">
  3. <html xmlns=“http://www.w3.org/1999/xhtml”>
  4. <head>
  5.   <meta http-equiv=“Content-Type” content=“text/html; charset=iso-8859-1″ />
  6.   <title>Test encoding</title>
  7. </head>
  8. <body>
  9.   <form id=“Main” runat=“server”>
  10.     <asp:ScriptManager ID=“ScriptManager” runat=“server” EnablePartialRendering=“true”></asp:ScriptManager>
  11.                
  12.     <h1>ISO-8859-1 / UTF-8 encoding problem in UpdatePanel</h1>
  13.                
  14.     Reguar submit button:<br />
  15.     <input type=“submit” value=“Submit” /><br />
  16.     <hr />
  17.                
  18.     Submit button in an UpdatePanel:<br />
  19.     <asp:UpdatePanel ID=“UpdatePanel3″ runat=“server” RenderMode=“inline”>
  20.       <ContentTemplate>
  21.         <input type=“submit” value=“Submit” />
  22.       </ContentTemplate>
  23.     </asp:UpdatePanel><br />
  24.     <hr />
  25.  
  26.     Result from server:<br />
  27.     <asp:UpdatePanel ID=“UpdatePanel2″ runat=“server” RenderMode=“inline”>
  28.       <ContentTemplate>
  29.         <asp:Label ID=“Result” Text=“” runat=“server” />
  30.       </ContentTemplate>
  31.     </asp:UpdatePanel><br />
  32.     <hr />
  33.  
  34.     String value to send to the server:<br />
  35.     <asp:TextBox ID=“TextBoxGroupName” Text=“æøå” runat=“server”></asp:TextBox> (do not modify!)
  36.   </form>
  37. </body>
  38. </html>

TestEncoding.aspx.cs

  1. using System;
  2. using System.Data;
  3. using System.Configuration;
  4. using System.Collections;
  5. using System.Web;
  6. using System.Web.Security;
  7. using System.Web.UI;
  8. using System.Web.UI.WebControls;
  9. using System.Web.UI.WebControls.WebParts;
  10. using System.Web.UI.HtmlControls;
  11. using System.Xml;
  12. using System.Xml.Xsl;
  13. using System.IO;
  14. using System.Text;
  15.  
  16. public partial class TestEncoding : System.Web.UI.Page
  17. {
  18.   protected void Page_Load(object sender, EventArgs e)
  19.     {
  20.       if (IsPostBack)
  21.       {
  22.       string t = TextBoxGroupName.Text;
  23.       int l = t.Length;
  24.       if (l == 6)
  25.       {
  26.         Result.Text = “<b style=’color:red’>Error: Got an UTF-8 server reply: ‘” + t + “‘.</b>”;
  27.       }
  28.       else
  29.       {
  30.         Result.Text = “<b style=’color:#060;’>Approved: Got an ISO-8859-1 server reply: ‘” + t + “‘.</b>”;
  31.       }
  32.     }
  33.     else
  34.     {
  35.       Result.Text = “<i>Press one of the submit buttons.</i>”;
  36.     }
  37.   }
  38. }

When you run this example pressing the regular submit button will result in the server receiving the expected string value “æøå”. Pressing the submit button inside the UpdatePanel however, posts the UTF-8 encoded string value “æøå” to the server.

Google + hard earned knowledge to the rescue

Now googling payed off - and I was able to locate this official bug report from Microsoft regarding non UTF-8 encoding and UpdatePanel.

The solution is to insert the following javascript on all form pages in your web application where the problem could potentially occur:

The bug solving hack

  1. <script type=“text/javascript”>
  2. <!–
  3. function pageLoad(sender, args) {
  4.   if (!args.get_isPartialLoad()) {
  5.     Sys.WebForms.PageRequestManager.getInstance().add_beginRequest(OnBeginRequest);
  6.   }
  7. }
  8. function OnBeginRequest(sender, args) {
  9.   args.get_request().get_headers()[“Content-Type”] = “application/x-www-form-urlencoded; charset=utf-8″;
  10. }
  11. //–>
  12. </script>

Throw that piece of code into the head section of the TestEncoding.aspx file, and voilá, the value is interpretted correctly by the Microsoft .NET AJAX framework.

I sure hope my article will save some of your time, as I am all to clearly aware of how much time I have lost in the process of tracking down the problem. Maybe I even ought to invoice the Redmond company.

1 kommentar »

  1. Thank you!

    Very good post. Saved me for some hours. (I’m using a master file, so I only needed to paste the script once).

    Kommentar skrevet af Terje
    D. 09/01-2008 Kl. 17:03:46

RSS feed for kommentarer til dette indlæg. TrackBack URI

Send kommentar

*
For at bevise at du er en person, og ikke en spam-robot, skal du indtaste dette kodeord. Bemærk at kodeordet altid starter eller slutter med et tal. Klik på billedet for at få kodeordet læst op.
Click to hear an audio file of the anti-spam word